OSError：[Errno 24] Python 导入时打开的文件过多答案

【问题标题】：OSError: [Errno 24] Too many open files on python importOSError：[Errno 24] Python 导入时打开的文件过多
【发布时间】：2019-04-11 22:21:26
【问题描述】：

我有一个从 c 代码调用 python 的测试环境：

system("flows_parser.py ....");

长时间运行后，我每次都会收到此错误：

Traceback (most recent call last):
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/flows_parser.py", line 32, in <module>
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/../../shared/proto_steps/common/generated_py/proto_steps_common_steps_defines_pb2.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.5/dist-packages/google/protobuf/reflection.py'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/lib/python3/dist-packages/apport/__init__.py'

Original exception was:
Traceback (most recent call last):
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/flows_parser.py", line 32, in <module>
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/../../shared/proto_steps/common/generated_py/proto_steps_common_steps_defines_pb2.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.5/dist-packages/google/protobuf/reflection.py'

flows_parser.py proto_steps_common_steps_defines_pb2 导入中的第 32 行 proto_steps_common_steps_defines_pb2.py 第9行也是反射的导入。

我使用的是 Ubuntu 16.4 输出：sysctl -A | grep shm

kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
sysctl: reading key "net.ipv6.conf.all.stable_secret"
sysctl: reading key "net.ipv6.conf.default.stable_secret"
sysctl: reading key "net.ipv6.conf.enp0s3.stable_secret"
sysctl: reading key "net.ipv6.conf.enp0s8.stable_secret"
sysctl: reading key "net.ipv6.conf.lo.stable_secret"
vm.hugetlb_shm_group = 0

所以 shmmax 应该不是问题。有人可以帮忙吗？

【问题讨论】：

ulimit -n 的输出是什么？
$ ulimit -n : 1024
尝试将此软限制增加到ulimit -Hf 运行ulimit -f $(ulimit -Hf) 报告的值。您的程序可能存在文件描述符泄漏。
我想不出任何在 python 退出后仍然打开的文件，我回到 c 程序
有什么方法可以查看我的进程打开了多少文件描述符？

标签： linux python-3.x

【解决方案1】：

感谢里卡多布兰科！它是来自 c 代码的描述符韭菜，与 python 无关。我创建了管道，完成后没有关闭它们。我找到了描述符韭菜并插入它，现在它工作正常。

【讨论】：