Pandas：我正在尝试打开存储在 ftp 站点上的 zip 文件中的特定 .txt 文件

Question

我需要将来自this ftp 站点的每个 zip 文件中的每个 produkt_monat_Monatswerte_18910101_20110331_00003.txt 文件连接到一个框架中。

这是我目前使用的代码：

    import pandas as pd
    from pandas.io.parsers import *
    import glob
    import requests
    from zipfile import ZipFile
    import urllib.request as ur


    years = 'produkt_monat_Monatswerte_*.txt'

names = pd.DataFrame()
for year in years:
    path ="ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate/monthly/kl/historical/monatswerte_?????_????????_????????_hist.zip").read()
    frame = pd.read_csv(path, names=columns)

    frame['year'] = year
    names = names.concat(frame, ignore_index=True)

它给了我以下错误：

 File "<ipython-input-25-d57a1d77ecc6>", line 5
    path ="ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate/monthly/kl/historical/monatswerte_?????_????????_????????_hist.zip")

Answer 1

问题是您不能让 pandas 从 Zip 中提取内部文件。 试试下面的代码：

import pandas as pd
from ftplib import FTP
import os
from zipfile import ZipFile
from io import BytesIO

f_root = 'ftp-cdc.dwd.de'
zips_path = '/pub/CDC/observations_germany/climate/monthly/kl/historical/'

ftp = FTP(f_root)

ftp.login()

ftp.cwd(zips_path)

paths = [p[0] for p in ftp.mlsd('.') if p[0].endswith('.zip')]

dfs = []

for path in paths:
    buf = BytesIO()
    ftp.retrbinary("RETR " + path, lambda block: buf.write(block))
    z = ZipFile(buf)

    zi = list(filter(lambda x: x.filename.startswith('produkt'), z.filelist))[0]
    df = pd.read_csv(BytesIO(z.read(zi.filename)), sep=';', encoding="cp1252")
    dfs.append(df)

final = pd.concat(dfs)