为什么上下文管理器不关闭文件描述符？

Question

我正在尝试创建一个使用mmap 的上下文管理器，它本身就是一个上下文管理器。最初我遇到了一个愚蠢的打开文件问题Why isn't mmap closing associated file (getting PermissionError: [WinError 32])?，并且很快得到了一个答案，为什么它没有按预期工作。

鉴于这些信息，我尝试了两种不同的方法来纠正该问题，但都没有奏效。

第一种方法是使用contextlib的@contextmanager装饰器：

from contextlib import contextmanager
import os
import mmap

#contextmanager
def memory_map(filename, access=mmap.ACCESS_WRITE):
    size = os.path.getsize(filename)
    fd = os.open(filename, os.O_RDWR)
    print('about to yield')
    with mmap.mmap(fd, size, access=access) as m:
        yield m
    print('in finally clause')
    os.close(fd)  # Close the associated file descriptor.

test_filename = 'data'

# First create the test file.
size = 1000000
with open(test_filename, 'wb') as f:
     f.seek(size - 1)
     f.write(b'\x00')

# Read and modify mmapped file in-place.
with memory_map(test_filename) as m:  # Causes AttributeError: __enter__
    print(len(m))
    print(m[0:10])
    # Reassign a slice.
    m[0:11] = b'Hello World'

# Verify that changes were made
print('reading back')
with open(test_filename, 'rb') as f:
     print(f.read(11))

# Delete test file.
# Causes:
# PermissionError: [WinError 32] The process cannot access the file because it
# is being used by another process: 'data'
os.remove(test_filename)

但结果是：

Traceback (most recent call last):
  File "memory_map.py", line 27, in <module>
    with memory_map(test_filename) as m:  # Causes AttributeError: __enter__
AttributeError: __enter__

在下一次尝试中，我尝试显式创建上下文管理器类：

import os
import mmap

class MemoryMap:
    def __init__(self, filename, access=mmap.ACCESS_WRITE):
        print('in MemoryMap.__init__')
        size = os.path.getsize(filename)
        self.fd = os.open(filename, os.O_RDWR)
        self.mmap = mmap.mmap(self.fd, size, access=access)

    def __enter__(self):
        print('in MemoryMap.__enter__')
        return self.mmap

    def __exit__(self, exc_type, exc_value, traceback):
        print('in MemoryMap.__exit__')
        os.close(self.fd)  # Close the associated file descriptor.
        print('  file descriptor closed')


test_filename = 'data'

# First create the test file.
size = 1000000
with open(test_filename, 'wb') as f:
     f.seek(size - 1)
     f.write(b'\x00')

# Read and modify mmapped file in-place.
with MemoryMap(test_filename) as m:
    print(len(m))
    print(m[0:10])
    # Reassign a slice.
    m[0:11] = b'Hello World'

# Verify that changes were made
print('reading back')
with open(test_filename, 'rb') as f:
     print(f.read(11))

# Delete test file.
# Causes PermissionError: [WinError 32] The process cannot access the file
# because it is being used by another process: 'data'
os.remove(test_filename)

这让它更进一步，但PermissionError 又回来了——这真的让我很困惑，因为文件描述符在那个版本中被关闭了，正如你在生成的输出中看到的那样：

in MemoryMap.__init__
in MemoryMap.__enter__
1000000
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
in MemoryMap.__exit__
  file descriptor closed
reading back
b'Hello World'
Traceback (most recent call last):
  File "memory_map2.py", line 47, in <module>
    os.remove(test_filename)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'data'

看来我又被困住了。关于出了什么问题（以及如何解决它）的任何想法？另外，如果它们都可以修复，如果您有意见，哪个更好？

解决方案

两个 sn-ps 都有错误。这首先是一个简单的印刷错误。 contextmanger 装饰器被注释掉了。应该是：

@contextmanager  # Leading "#" changed to "@".
def memory_map(filename, access=mmap.ACCESS_WRITE):
    size = os.path.getsize(filename)
    fd = os.open(filename, os.O_RDWR)
    ...

第二个原因是mmap 本身 没有在__exit__() 方法中关闭，只是关联的文件描述符。我从来没有想到过，因为引发的异常与第一种情况相同。

    def __exit__(self, exc_type, exc_value, traceback):
        print('in MemoryMap.__exit__')
        self.mmap.close()  # ADDED.
        os.close(self.fd)  # Close the associated file descriptor.
        print('  file descriptor closed')

Answer 1

如果您第二次尝试，您需要关闭内存映射文件：

def __exit__(self, exc_type, exc_value, traceback):
    self.mm.close()
    print('in MemoryMap.__exit__')
    os.close(self.fd)  # Close the associated file descriptor.
    print('  file descriptor closed')