【问题标题】:Why python script with threading lib eats 100% CPU?为什么带有线程库的 python 脚本会占用 100% 的 CPU?
【发布时间】:2020-09-06 12:13:47
【问题描述】:

我有一些客户端脚本,它连接到套接字并等待数据。 在服务器端有带有 json 包装器的 ZeroMQ 库。

所以,实际上,它是一个 json 数据格式的套接字连接。它是来自 metatrader 的股票市场数据。 所以我每分钟等待一条数据线。因为我的时间框架是 M1。

问题是为什么这个脚本会占用 100% cpu?

大多数时候它应该等待数据并且什么都不做。我觉得脚本中可能有一些错误。但是我还没有技巧。 请告诉我应该在哪里寻找问题。

这是一个脚本:



    import zmq
    import threading
    class MTraderAPI:
        def __init__(self, host=None):
            self.HOST = host or 'localhost'
            self.SYS_PORT = 15555       # REP/REQ port
            self.DATA_PORT = 15556      # PUSH/PULL port
            self.LIVE_PORT = 15557      # PUSH/PULL port
            self.EVENTS_PORT = 15558    # PUSH/PULL port
    
            # ZeroMQ timeout in seconds
            sys_timeout = 1
            data_timeout = 10
    
            # initialise ZMQ context
            context = zmq.Context()
    
            # connect to server sockets
            try:
                self.sys_socket = context.socket(zmq.REQ)
                # set port timeout
                self.sys_socket.RCVTIMEO = sys_timeout * 1000
                self.sys_socket.connect('tcp://{}:{}'.format(self.HOST, self.SYS_PORT))
    
                self.data_socket = context.socket(zmq.PULL)
                # set port timeout
                self.data_socket.RCVTIMEO = data_timeout * 1000
                self.data_socket.connect('tcp://{}:{}'.format(self.HOST, self.DATA_PORT))
            except zmq.ZMQError:
                raise zmq.ZMQBindError("Binding ports ERROR")
    
        def _send_request(self, data: dict) -> None:
            """Send request to server via ZeroMQ System socket"""
            try:
                self.sys_socket.send_json(data)
                msg = self.sys_socket.recv_string()
                # terminal received the request
                assert msg == 'OK', 'Something wrong on server side'
            except AssertionError as err:
                raise zmq.NotDone(err)
            except zmq.ZMQError:
                raise zmq.NotDone("Sending request ERROR")
    
        def _pull_reply(self):
            """Get reply from server via Data socket with timeout"""
            try:
                msg = self.data_socket.recv_json()
            except zmq.ZMQError:
                raise zmq.NotDone('Data socket timeout ERROR')
            return msg
    
        def live_socket(self, context=None):
            """Connect to socket in a ZMQ context"""
            try:
                context = context or zmq.Context.instance()
                socket = context.socket(zmq.PULL)
                socket.connect('tcp://{}:{}'.format(self.HOST, self.LIVE_PORT))
            except zmq.ZMQError:
                raise zmq.ZMQBindError("Live port connection ERROR")
            return socket
    
        def streaming_socket(self, context=None):
            """Connect to socket in a ZMQ context"""
            try:
                context = context or zmq.Context.instance()
                socket = context.socket(zmq.PULL)
                socket.connect('tcp://{}:{}'.format(self.HOST, self.EVENTS_PORT))
            except zmq.ZMQError:
                raise zmq.ZMQBindError("Data port connection ERROR")
            return socket
    
        def construct_and_send(self, **kwargs) -> dict:
            """Construct a request dictionary from default and send it to server"""
    
            # default dictionary
            request = {
                "action": None,
                "actionType": None,
                "symbol": None,
                "chartTF": None,
                "fromDate": None,
                "toDate": None,
                "id": None,
                "magic": None,
                "volume": None,
                "price": None,
                "stoploss": None,
                "takeprofit": None,
                "expiration": None,
                "deviation": None,
                "comment": None
            }
    
            # update dict values if exist
            for key, value in kwargs.items():
                if key in request:
                    request[key] = value
                else:
                    raise KeyError('Unknown key in **kwargs ERROR')
    
            # send dict to server
            self._send_request(request)
    
            # return server reply
            return self._pull_reply()
    
    api = MTraderAPI()
    print(api.construct_and_send(action="CONFIG", symbol="XBRUSD.c", chartTF="TICK"))
    
    def _t_livedata():
        socket = api.live_socket()
        while True:
            try:
                last_candle = socket.recv_json()
            except zmq.ZMQError:
                raise zmq.NotDone("Live data ERROR")
            print(last_candle)
    
    
    def _t_streaming_events():
        socket = api.streaming_socket()
        while True:
            try:
                trans = socket.recv_json()
                request, reply = trans.values()
            except zmq.ZMQError:
                raise zmq.NotDone("Streaming data ERROR")
            print(request)
            print(reply)
    
    
    
    t = threading.Thread(target=_t_livedata, daemon=True)
    t.start()
    
    t = threading.Thread(target=_t_streaming_events, daemon=True)
    t.start()
    
    while True:
        pass

【问题讨论】:

  • 为线程提供正确的唯一名称,并将末尾的循环替换为t1.join(); t2.join()

标签: python json multithreading zeromq


【解决方案1】:

这是你的:

while True:
    pass

最后 - 它的全部意义在于消耗 CPU。使用time.sleep 添加一个小睡眠,CPU 使用率会降低。或者,如果您只等待线程完成,只需加入它们,它将阻塞直到它们执行完成。

例如:

from time import sleep

while True:
    sleep(0.1)

或者如果你将你的线程对象收集到一个列表中,你应该:

for t in threads:
    t.join()

有关更多信息,请参阅following post

【讨论】:

  • 刚刚意识到并尝试 sleep(0.01) 现在它根本没有负载。这是线程使用的正确方法吗?我的意思是用最小的延迟填充一个空循环。或者它只是一种解决方法,可能有一种聪明的 Pythonic 方式?
  • @navy 检查加入线程的第二个选项,如果您正在等待线程完成,这是最好的解决方案
猜你喜欢
  • 2023-03-06
  • 2014-09-16
  • 2012-03-10
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2017-01-27
  • 2020-03-22
相关资源
最近更新 更多