使用 paramiko 的 Python sftp 错误答案

【问题标题】：Python sftp error with paramiko使用 paramiko 的 Python sftp 错误
【发布时间】：2016-11-14 16:54:12
【问题描述】：

为什么我在调用 get 函数时使用 paramiko 出现连接断开错误？

class AllowAnythingPolicy(paramiko.MissingHostKeyPolicy):
    def missing_host_key(self, client, hostname, key):
        return

client = paramiko.SSHClient()
client.set_missing_host_key_policy(AllowAnythingPolicy())
client.connect('', username='',password='')
sftp.get('','')

我有一个 70 mb 的文件，在出现错误后函数下载 20 mb。当大小文件小于 20mb 时，此功能可以正常工作

这是 paramiko 日志文件：

DEB [20161115-10:25:47.792] thr=1   paramiko.transport: starting thread (client mode): 0x472a3d0
DEB [20161115-10:25:47.793] thr=1   paramiko.transport: Local version/idstring: SSH-2.0-paramiko_2.0.2
DEB [20161115-10:25:47.793] thr=1   paramiko.transport: Remote version/idstring: SSH-2.0-SshServer
INF [20161115-10:25:47.794] thr=1   paramiko.transport: Connected (version 2.0, client SshServer)
DEB [20161115-10:25:47.795] thr=1   paramiko.transport: kex algos:['ecdh-sha2-nistp256', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp521', 'diffie-hellman-group-exchange-sha1', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group14-sha1', 'diffie-hellman-group1-sha1'] server key:['ssh-rsa'] client encrypt:['aes256-ctr', 'aes256-cbc'] server encrypt:['aes256-ctr', 'aes256-cbc'] client mac:['hmac-sha1', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-ripemd160', 'hmac-ripemd160@openssh.com'] server mac:['hmac-sha1', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-ripemd160', 'hmac-ripemd160@openssh.com'] client compress:['none'] server compress:['none'] client lang:['en-US'] server lang:['en-US'] kex follows?False
DEB [20161115-10:25:47.795] thr=1   paramiko.transport: Kex agreed: diffie-hellman-group1-sha1
DEB [20161115-10:25:47.796] thr=1   paramiko.transport: Cipher agreed: aes256-ctr
DEB [20161115-10:25:47.796] thr=1   paramiko.transport: MAC agreed: hmac-sha2-256
DEB [20161115-10:25:47.796] thr=1   paramiko.transport: Compression agreed: none
DEB [20161115-10:25:48.054] thr=1   paramiko.transport: kex engine KexGroup1 specified hash_algo <built-in function openssl_sha1>
DEB [20161115-10:25:48.054] thr=1   paramiko.transport: Switch to new keys ...
DEB [20161115-10:25:48.057] thr=1   paramiko.transport: userauth is OK
INF [20161115-10:25:48.137] thr=1   paramiko.transport: Authentication (password) successful!
DEB [20161115-10:25:57.677] thr=2   paramiko.transport: [chan 0] Max packet in: 32768 bytes
DEB [20161115-10:25:57.680] thr=1   paramiko.transport: [chan 0] Max packet out: 32768 bytes
DEB [20161115-10:25:57.681] thr=1   paramiko.transport: Secsh channel 0 opened.
DEB [20161115-10:25:57.682] thr=1   paramiko.transport: [chan 0] Sesch channel 0 request ok
INF [20161115-10:25:57.684] thr=2   paramiko.transport.sftp: [chan 0] Opened sftp connection (server version 3)
DEB [20161115-10:25:57.685] thr=2   paramiko.transport.sftp: [chan 0] stat(b'/GEO/OUT')
DEB [20161115-10:25:57.688] thr=2   paramiko.transport.sftp: [chan 0] normalize(b'/GEO/OUT')
DEB [20161115-10:25:57.690] thr=2   paramiko.transport.sftp: [chan 0] listdir(b'/GEO/OUT/.')
DEB [20161115-10:26:02.008] thr=2   paramiko.transport.sftp: [chan 0] stat(b'/GEO/OUT/test.csv')
DEB [20161115-10:26:02.012] thr=2   paramiko.transport.sftp: [chan 0] open(b'/GEO/OUT/test.csv', 'rb')
DEB [20161115-10:26:02.016] thr=2   paramiko.transport.sftp: [chan 0] open(b'/GEO/OUT/test.csv', 'rb') -> b'2f47454f2f4f55542f746573742e637376'
DEB [20161115-10:28:10.626] thr=1   paramiko.transport: EOF in transport thread
DEB [20161115-10:28:10.626] thr=2   paramiko.transport.sftp: [chan 0] close(b'2f47454f2f4f55542f746573742e637376')

和 Python 错误：

EOFError                                  Traceback (most recent call last)
C:\Anaconda3\lib\site-packages\paramiko\sftp_client.py in _read_response(self, waitfor)
    759             try:
--> 760                 t, data = self._read_packet()
    761             except EOFError as e:

C:\Anaconda3\lib\site-packages\paramiko\sftp.py in _read_packet(self)
    172     def _read_packet(self):
--> 173         x = self._read_all(4)
    174         # most sftp servers won't accept packets larger than about 32k, so

C:\Anaconda3\lib\site-packages\paramiko\sftp.py in _read_all(self, n)
    158             if len(x) == 0:
--> 159                 raise EOFError()
    160             out += x

EOFError: 

During handling of the above exception, another exception occurred:

SSHException                              Traceback (most recent call last)
<ipython-input-49-b52d34c6bd07> in <module>()
----> 1 sftp.get('/GEO/OUT/test.csv','ESRI_OUT/te.csv')

C:\Anaconda3\lib\site-packages\paramiko\sftp_client.py in get(self, remotepath, localpath, callback)
    719         """
    720         with open(localpath, 'wb') as fl:
--> 721             size = self.getfo(remotepath, fl, callback)
    722         s = os.stat(localpath)
    723         if s.st_size != size:

C:\Anaconda3\lib\site-packages\paramiko\sftp_client.py in getfo(self, remotepath, fl, callback)
    697             fr.prefetch(file_size)
    698             return self._transfer_with_callback(
--> 699                 reader=fr, writer=fl, file_size=file_size, callback=callback
    700             )
    701 

C:\Anaconda3\lib\site-packages\paramiko\sftp_client.py in _transfer_with_callback(self, reader, writer, file_size, callback)
    596         size = 0
    597         while True:
--> 598             data = reader.read(32768)
    599             writer.write(data)
    600             size += len(data)

C:\Anaconda3\lib\site-packages\paramiko\file.py in read(self, size)
    209                 read_size = max(self._bufsize, read_size)
    210             try:
--> 211                 new_data = self._read(read_size)
    212             except EOFError:
    213                 new_data = None

C:\Anaconda3\lib\site-packages\paramiko\sftp_file.py in _read(self, size)
    163         size = min(size, self.MAX_REQUEST_SIZE)
    164         if self._prefetching:
--> 165             data = self._read_prefetch(size)
    166             if data is not None:
    167                 return data

C:\Anaconda3\lib\site-packages\paramiko\sftp_file.py in _read_prefetch(self, size)
    143             if self._prefetch_done or self._closed:
    144                 break
--> 145             self.sftp._read_response()
    146             self._check_exception()
    147         if offset is None:

C:\Anaconda3\lib\site-packages\paramiko\sftp_client.py in _read_response(self, waitfor)
    760                 t, data = self._read_packet()
    761             except EOFError as e:
--> 762                 raise SSHException('Server connection dropped: %s' % str(e))
    763             msg = Message(data)
    764             num = msg.get_int()

SSHException: Server connection dropped:

【问题讨论】：

您可以使用独立的 SFTP 客户端下载文件吗？
是的，winscp 工作！
向我们展示 WinSCP 和 paramiko log files。
当我使用get函数时：EOFError：在处理上述异常过程中，又发生了一个异常：SSHException SSHException：服务器连接断开：
请提供日志文件！当然，在问题中包含完整的异常消息（有“编辑”链接）。

标签： python sftp paramiko

【解决方案1】：

我的问题的解决方案是：

tr = client.get_transport()
tr.default_max_packet_size = 100000000
tr.default_window_size = 100000000

【讨论】：

【解决方案2】：

我猜你得到资源错误？随时用实际错误更新问题。

如果错误与资源有关，我会尝试使用 open 而不是 get，我会使用 open 和 read，如下所示：

class AllowAnythingPolicy(paramiko.MissingHostKeyPolicy):
    def missing_host_key(self, client, hostname, key):
        return

client = paramiko.SSHClient()
client.set_missing_host_key_policy(AllowAnythingPolicy())
client.connect('', username='',password='')
file = sftp.open('','')
file.read # Will get you the file.

这是由于 SFTP 不能真正流式传输文件的问题。

另一种可能性是设置传输文件的块大小，如下所示：

remotefile = sftp.open('','')
with remotefile.file(remote_file_path, mode='w') as sftpfile:
    sftpfile.MAX_REQUEST_SIZE = 1024
    sftpfile.write(data)

【讨论】：