使用python多处理同时写入文件

Question

我正在开发一个项目，其中我使用大约 6 个传感器输入 Beaglebone Black，它将数据保存到 6 个不同的文件中连续。通过另一个 SO 问题 (https://stackoverflow.com/a/36634587/2615940)，我了解到多处理模块会为我执行此操作，但是在运行我的新代码时，我只获得 1 个文件而不是 6 个。如何修改此代码以获得所需的 6结果文件？

*我已经根据下面 skrrgwasme 的建议编辑了我的文件以包含 Manager，但现在代码运行了，但什么也没产生。没有错误，没有文件。只是跑。

代码：

import Queue
import multiprocessing
import time

def emgacq(kill_queue, f_name, adcpin):
     with open(f_name, '+') as f:
        while True:
            try:
                val = kill_queue.get(block = False)
                if val == STOP:
                    return
            except Queue.Empty:
                pass
            an_val = ADC.read(adcpin) * 1.8
            f.write("{}\t{}\n".format(ms, an_val))
def main():    
    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    #Multiprcessing settings
    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()            

    #All the arguments we need run thru emgacq()
    arg_list = [
        (kill_queue, 'HamLeft', 'AIN1'),
        (kill_queue, 'HamRight', 'AIN2'),
        (kill_queue, 'QuadLeft', 'AIN3'),
        (kill_queue, 'QuadRight', 'AIN4'),
        (kill_queue, 'GastLeft', 'AIN5'),
        (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a) 

    try: 
        while True:
            time.sleep(60) 
    except KeyboardInterrupt:
        for a in arg_list:
            kill_queue.put(STOP)
        pool.close()
        pool.join()
        raise f.close()

if __name__ == "__main__":
    main()

Answer 1

您的主要问题是您的子流程函数的参数列表不正确：

f_list = [
    emgacq(kill_queue, 'HamLeft', 'AIN1'), 
    # this calls the emgacq function right here - blocking the rest of your 
    # script's execution

另外，您的apply_async 呼叫是错误的：

for f in f_list:
    pool.apply_async(f, args=(kill_queue)) 
    # f is not a function here - the arguments to the apply_async function
    # should be the one function you want to call followed by a tuple of 
    # arguments that should be provided to it

您想要这个，其中还包括一个用于队列的manager（请参阅https://stackoverflow.com/a/9928191/2615940）并将您的所有代码放入一个main 函数中：

# put your imports here
# followed by the definition of the emgacq function

def main():

    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()

    arg_list = [
            (kill_queue, 'HamLeft', 'AIN1'), 
            (kill_queue, 'HamRight', 'AIN2'),
            (kill_queue, 'QuadLeft', 'AIN3'),
            (kill_queue, 'QuadRight', 'AIN4'),
            (kill_queue, 'GastLeft', 'AIN5'),
            (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a)
        # this will call the emgacq function with the arguments provided in "a"

if __name__ == "__main__":
    # you want to have all of your code in a function, because the workers
    # will start by importing the main module they are executing from,
    # and you don't want them to execute that code all over again
    main()