.apply_on_packets 的超时时间是不可预测的吗？

Question

以下代码在使用以下advice 时会产生不可预知的结果：

import pyshark
import pandas as pd
import asyncio

def ProcessPackets(packet):
    global packet_list
    packet_version = packet.layers[1].version
    layer_name = packet.layers[2].layer_name
    packet_list.append([packet_version, layer_name, packet.length, packet.sniff_time])

def Capture(timeOrPath):
    global packet_list
    packet_list=[]
    try:
        timeout=int(timeOrPath)
        capture = pyshark.LiveCapture()          
        capture.apply_on_packets(ProcessPackets, timeout=timeout)
    except asyncio.TimeoutError:
        pass
    except ValueError:
        capture = pyshark.FileCapture(timeOrPath)
        capture.load_packets()
        capture.apply_on_packets(ProcessPackets)
    data = pd.DataFrame(packet_list, columns=['vIP', 'protocol', 'length','timestamp']) 
    print(data['timestamp'].iloc[-1]-data['timestamp'].iloc[0])

def main(): 
    Capture(6)

if __name__ == '__main__':
    main()

有时计算的时间会超过给定的超时时间。 （timestamp 是packet.sniff_time）

Answer 1

2021 年 6 月 3 日更新

---

在对此捕获延迟问题进行一些研究后，我确定该问题可能与 pyshark 等待 dumpcap 加载有关。 dumpcap 在 LiveCapture 模式下加载

  def _get_dumpcap_parameters(self):
        # Don't report packet counts.
        params = ["-q"]
        if self._get_tshark_version() < LooseVersion("2.5.0"):
            # Tshark versions older than 2.5 don't support pcapng. This flag forces dumpcap to output pcap.
            params += ["-P"]
        if self.bpf_filter:
            params += ["-f", self.bpf_filter]
        if self.monitor_mode:
            params += ["-I"]
        for interface in self.interfaces:
            params += ["-i", interface]
        # Write to STDOUT
        params += ["-w", "-"]
        return params

    async def _get_tshark_process(self, packet_count=None, stdin=None):
        read, write = os.pipe()

        dumpcap_params = [get_process_path(process_name="dumpcap", tshark_path=self.tshark_path)] + self._get_dumpcap_parameters()

        self._log.debug("Creating Dumpcap subprocess with parameters: %s" % " ".join(dumpcap_params))
        dumpcap_process = await asyncio.create_subprocess_exec(*dumpcap_params, stdout=write,
                                                               stderr=self._stderr_output())
        self._created_new_process(dumpcap_params, dumpcap_process, process_name="Dumpcap")

        tshark = await super(LiveCapture, self)._get_tshark_process(packet_count=packet_count, stdin=read)
        return tshark

上面的代码在我的系统上启动它：

 /usr/local/bin/dumpcap -q -i en0 -w -

还有这个：

/usr/local/bin/tshark -l -n -T pdml -r -

我尝试将一些自定义参数传递给 LiveCapture

capture = pyshark.LiveCapture(interface='en0', custom_parameters=["-q", "--no-promiscuous-mode", "-l"])

但仍有大约 1/2 秒的延迟。

10.015577793121338
0 days 00:00:09.371264

在 dumpcap 文档中有一个 -a 模式，它允许持续时间超时，但我无法将该参数传递给 pyshark不会导致错误。

Tshark也有-a模式，但也会导致pyshark

内部报错

capture = pyshark.LiveCapture(interface='en0', override_prefs={'': '-r'}, custom_parameters={'': '-a duration:20'})

可能有办法在 pyshark 代码库中修改超时参数，以允许 -a 模式。为此需要进行一些测试，我目前没有时间进行。

我与 pyshark 的开发人员打开了issue on this problem。

2021 年 6 月 2 日原帖

---

我重新编写了您的代码以将提取的项目写入 pandas 数据框。如果这不是您想要的，请根据您的确切要求更新您的问题。

import pyshark
import asyncio
import pandas as pd

packet_list = []


def process_packets(packet):
    global packet_list
    try:
        packet_version = packet.layers[1].version
        layer_name = packet.layers[2].layer_name
        packet_list.append([packet_version, layer_name, packet.length, str(packet.sniff_time)])
    except AttributeError:
        pass


def capture_packets(timeout):
    capture = pyshark.LiveCapture(interface='en0')
    try:
        capture.apply_on_packets(process_packets, timeout=timeout)
    except asyncio.TimeoutError:
        pass
    finally:
        return packet_list


def main():
    capture_packets(6)
    df = pd.DataFrame(packet_list, columns=['packet version', 'layer type', 'length', 'capture time'])
    print(df)
    # output 
          packet version layer type length                capture time
    0                 4        udp     75  2021-06-02 16:22:36.463805
    1                 4        udp     67  2021-06-02 16:22:36.517076
    2                 4        udp   1388  2021-06-02 16:22:36.706240
    3                 4        udp   1392  2021-06-02 16:22:36.706245
    4                 4        udp   1392  2021-06-02 16:22:36.706246
    truncated...


if __name__ == '__main__':
    main()