写模式下的负寻道

Question

我从 S3 下载了一个压缩文件并尝试解压缩：

with zipfile.ZipFile('/tmp/DataPump_10000838.zip', 'r') as zip_ref:
    testList = []
    for i in zip_ref.namelist():
        if (i.startswith("__MACOSX/") == False):
            val = '/tmp/'+i
            testList.append(val)

现在，我的/tmp 文件夹中有一些解压缩文件。我想打开这些文件，gzip 并移动到 S3 存储桶。解压文件的路径在testList

s3_filename = 'sample'

s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('testunzipping')    
    
    for i in testList:
        with contextlib.ExitStack() as stack:
            source_file = stack.enter_context(open(i , mode="rb"))
            destination_file = io.BytesIO()
            destination_file_gz = stack.enter_context(gzip.GzipFile(fileobj=destination_file, mode='wb'))
            while True:
                chunk = source_file.read(1024)
                if not chunk:
                    break
                destination_file_gz.write(chunk)
            destination_file_gz.seek(0)
            
            bucket.upload_fileobj(destination_file_gz, fileName)

我正在尝试这个，但我得到一个错误：

Response
{
  "errorMessage": "Negative seek in write mode",
  "errorType": "OSError",
  "requestId": "",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.9/importlib/__init__.py\", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n",
    "  File \"<frozen importlib._bootstrap>\", line 1030, in _gcd_import\n",
    "  File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n",
    "  File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n",
    "  File \"<frozen importlib._bootstrap>\", line 680, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 850, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 228, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 51, in <module>\n    destination_file_gz.seek(0)\n",
    "  File \"/var/lang/lib/python3.9/gzip.py\", line 384, in seek\n    raise OSError('Negative seek in write mode')\n"
  ]
}

我该如何解决这个问题？

Answer 1

Python 的 gzip 包装器不支持查找。不过，这可能并不重要，因为您要上传 gzip 的字节，而不是在 gzip 视图上查找所产生的解压缩字节。此外，您需要close gzip 文件，因为它不知道您已完成写入，并且可能包含要刷新的最终缓冲区：

    # Ensure the gzip wrapper is done writing its data
    destination_file_gz.close()

    # Seek to the start of the raw data, not the gzip'd version
    destination_file.seek(0)
    # And upload that raw data    
    bucket.upload_fileobj(destination_file, fileName)