python 多线程logger问题

2025-02-28 23:46:48
推荐回答(1个)
回答1:

因为logging是threadsafe的,但不是process-safe(应该没有这个词儿,只是为了便于理解)的。这段代码就是多个进程共同操作一个日志文件。这种情况下,logging的行为就很难说了。


我测试了一下,日志中大概几百行。而且,可以看到一些顺序错乱现象:

Fri, 08 Aug 2014 01:19:38 logging_in_multithread.py[line:40] theadWorking ERROR 2
FFri, 08 Aug 2014 01:19:36 logging_in_multithread.py[line:40] theadWorking ERROR 11(注意这里的FFri)


把代码这样改:

    for num in range(processNum):
        p = Process(target=processWorking, args=('2',))
        processs.append(p) 
        p.start()
        p.join()

 

还有其他方法,比如:为logging实现一个FileHandler,以使logging在multiple process的环境下也能正常工作。这是我从网上了解到的做法,自己还没实践过。


Python Manual中logging Cookbook中有这么一段话:

Logging to a single file from multiple processes

Although logging is thread-safe, and logging to a single file from multiple threads in a single process is supported, logging to a single file from multiple processes is not supported, because there is no standard way to serialize access to a single file across multiple processes in Python. If you need to log to a single file from multiple processes, one way of doing this is to have all the processes log to a SocketHandler, and have a separate process which implements a socket server which reads from the socket and logs to file. (If you prefer, you can dedicate one thread in one of the existing processes to perform this function.)

这段话中也提出了另外一种解决方案。