具有类方法和类变量的 Python 多处理

Question

我正在设置一个导出 PDF 文件的程序，现在我想通过使用 Python 的多处理模块和 map 函数来加速它（我使用的是 Python 3.7.9）。

现在我在将多处理应用到我的代码时遇到了麻烦。我试图将代码分解成它的基本要素，然后看起来像这样：

from multiprocessing import Pool


def callFunctionInClass(i):
    #calls the function inside the class which is supposed to be parallelized
    SomeClass().func1(i)
    

class SomeClass:
    
    def setupUi(self):
        # sets up the Graphical user interface
        return
        
    def func1(self, i):
        currentListItem = self.someList[i]
        #from here on out I create a PDF which then is exported
        
    def parallelizationFunction(self):
        #set up the pool
        pool = Pool()
        # range which the map function is supposed to iterate over
        someRange = list(range(0, len(self.someList)))
        pool.map(callFunctionInClass, someList)
        
        

if __name__ == "__main__":
    #calls the class and sets up the GUI
    ui = SomeClass()
    ui.setupUi()

在类内部初始化池，并使用 map 调用顶级方法，然后应该调用我想在其上使用多进程处理的类方法。 现在已经在这里提出了类似的问题，我尝试根据答案采用，但我仍然遇到错误（主要是酸洗错误）。

到目前为止，我尝试了几件事，例如使用 pathos ProcessingPool 将 self 传递给顶级方法，但我也无法使其正常工作。我的问题是如何将类值传递给其范围之外的方法而不会遇到酸洗错误（也许 multiprocessing.Value 可以完成工作）？ 我希望我的问题不是太模糊，否则我会尽力提供更充分的信息。 非常感谢！

Answer 1

如果没有类的对象，您无法从类外部访问方法。您可以在类外部创建一个列表并使用它，或者使用类变量创建一个静态方法

在类外使用列表

from multiprocessing import Pool

dataList = [{'data': 'a'}, {'data': 'b'}, {'data': 'c'}]


def callFunctionInClass(i):
    # calls the function inside the class which is supposed to be parallelized
    SomeClass().func1(i)


class SomeClass:

    def setupUi(self):
        print('UI')
        # sets up the Graphical user interface
        return

    def func1(self, i):
        currentListItem = dataList[i]
        print(currentListItem)
        # from here on out I create a PDF which then is exported

    def parallelizationFunction(self):
        # set up the pool
        pool = Pool()
        # range which the map function is supposed to iterate over
        someRange = list(range(0, len(dataList)))
        pool.map(callFunctionInClass, someRange)


if __name__ == "__main__":
    # calls the class and sets up the GUI
    ui = SomeClass()
    ui.setupUi()
    ui.parallelizationFunction()

这只是将您的代码扩展为引用类外部的列表，它可以按预期工作

使用静态方法 (Static methods in Python)

from multiprocessing import Pool



def callFunctionInClass(i):
    # calls the function inside the class which is supposed to be parallelized
    SomeClass().func1(i)


class SomeClass:
    dataList = [{'data': 'a'}, {'data': 'b'}, {'data': 'c'}]

    def setupUi(self):
        print('UI')
        # sets up the Graphical user interface
        return

    @staticmethod
    def func1(i):
        currentListItem = SomeClass.dataList[i]
        print(currentListItem)
        # from here on out I create a PDF which then is exported

    def parallelizationFunction(self):
        # set up the pool
        pool = Pool()
        # range which the map function is supposed to iterate over
        someRange = list(range(0, len(SomeClass.dataList)))
        pool.map(callFunctionInClass, someRange)


if __name__ == "__main__":
    # calls the class and sets up the GUI
    ui = SomeClass()
    ui.setupUi()
    ui.parallelizationFunction()