【问题标题】:How to write data into text file using multithreading in python?如何在 python 中使用多线程将数据写入文本文件?
【发布时间】:2021-05-12 04:32:59
【问题描述】:

我想在技术上使用 python 中的多线程将数据附加到同一个文本文件中

这是我的代码_

import threading
my_list = ["a", "b", "c", "d", "e", "f"]
def f1():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f1 {item}")
       file.close()
def f2():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f2 {item}")
       file.close()
def f3():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f3 {item}")
       file.close()
def f4():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f4 {item}")
       file.close()
t1 = threading.Thread(target=f1)
t2 = threading.Thread(target=f2)
t3 = threading.Thread(target=f3)
t4 = threading.Thread(target=f4)
t1.start()
t2.start()
t3.start()
t4.start()
t1.join()
t2.join()
t3.join()
t4.join()

但是当我执行程序时,它并没有写任何东西,谁能告诉我该怎么做。

【问题讨论】:

  • 嗨,Christy,多线程处理单个资源(被另一个线程锁定的文件)有什么好处?
  • 试图同时从多个线程写入单个文件会使文件内容变得无法使用。如果您更改代码以便每个线程写入不同的文件(例如“order_log_1.txt”、“order_log_2.txt”等),OTOH,这将正常工作。
  • 没有 Jeremy 但我必须写在同一个文件中
  • Shawn 我听不懂你想告诉我什么...
  • 如果你必须写入同一个文件,那么你需要在所有线程之间共享一个文件句柄(而不是让每个线程打开自己单独的文件句柄),然后你'将需要使用互斥锁来保护该文件句柄的所有使用以避免竞争条件。这将序列化对文件的访问,以便一次只有一个线程可以写入。

标签: python multithreading python-multithreading writing


【解决方案1】:

您需要做到这一点,以便一次只有一个线程可以写入文件。这是通过让线程在文件上保存Lock 来完成的。一旦线程处理完文件,它就可以释放Lock,它会被等待它的线程之一占用。

这是一个如何使用锁的示例,以及Queue,因为您想要使用线程的原因通常是因为您想要对许多数据对象并行执行相同的操作。

import threading
import queue
import random
import time

tasks = queue.Queue()

output = open('output.txt', 'w')
output_lock = threading.Lock()

def worker(thread_number):
    while not tasks.empty():
        task = tasks.get()
        # perform long calculation here...
        time_to_spend = random.random()
        time.sleep(time_to_spend)
        result = task * task

        # now we have result, want to write it
        with output_lock:  # this will block until the lock is available
            print(thread_number, ': square of', task, 'is', result, '; took', time_to_spend, file=output)
        tasks.task_done()

for i in range(100):
    tasks.put(i)

for thread in range(8):
    threading.Thread(target=worker, args=(thread,)).start()

print('waiting for tasks to complete')
tasks.join()
print('done')

我得到的输出是:

6 : square of 6 is 36 ; took 0.02233345201885739
7 : square of 7 is 49 ; took 0.0352967148552743
4 : square of 4 is 16 ; took 0.1043699083780637
7 : square of 9 is 81 ; took 0.2158108589024338
1 : square of 1 is 1 ; took 0.3330501408298937
4 : square of 10 is 100 ; took 0.3564233912485101
5 : square of 5 is 25 ; took 0.8496825534757959
0 : square of 0 is 0 ; took 0.8807306770021203
4 : square of 13 is 169 ; took 0.4420943872313102
7 : square of 11 is 121 ; took 0.6772180132068408
5 : square of 14 is 196 ; took 0.1101644871869385
6 : square of 8 is 64 ; took 0.944739067078435
3 : square of 3 is 9 ; took 0.9699315957506418
2 : square of 2 is 4 ; took 0.9903787965119304
3 : square of 20 is 400 ; took 0.029847547355710158
1 : square of 12 is 144 ; took 0.696648284935379
...

【讨论】:

  • 这是一个不错的演示,它甚至显示处理数字的线程的 ID,但在实际编程中,我会使用 ThreadPoolExecutor。无论如何 +1...
  • 是的,我忘了这是一回事。你认为我应该编辑这个来添加一个演示吗?
  • 感谢您的分享,这是非常有用的信息!
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2020-11-25
  • 2017-11-03
  • 2013-02-22
  • 1970-01-01
相关资源
最近更新 更多