浏览器对 https 代理的响应不佳答案

【问题标题】：browser doesn't response well to https proxy浏览器对 https 代理的响应不佳
【发布时间】：2022-02-05 06:38:42
【问题描述】：

我已经构建了自己的 https 代理，当我向浏览器发送一些数据时，浏览器会毫无反应，而且会在很长一段时间后做出响应。基本上所有代理应该做的只是将消息转发到浏览器，获取响应并转发回客户端

代理的代码：

import socket
import select



serverSock = socket.socket()
serverSock.bind(('0.0.0.0', 8080))
serverSock.listen(3)
waiting_clients = {} # client : browser
users_dict = {}
open_clients = {}
browsers_clients = {} # browser : client

threading.Thread(target=browserCom).start()
while True:
    try:
        rlist, wlist, xlist = select.select(list(users_dict.keys()) + [serverSock], [], [], 0.3)
    except:
        pass
    else:
        for current_socket in rlist:
            if current_socket is serverSock:
                # new client
                client, address = serverSock.accept()
                print(f'{address} - connected to proxy')
                # add to dictionary
                users_dict[client] = address
                open_clients[address] = client
            else:
                # receive info
                receiving = True
                msg = bytearray()
                while receiving:
                    try:
                        data = current_socket.recv(1024)
                    except Exception as e:
                        print(e, 3)
                        if current_socket in users_dict.keys():
                            disconnect(users_dict[current_socket])

                        else:
                            current_socket.close()
                        break
                    else:
                        msg.extend(data)
                        # got the full msg
                        if len(data) < 1024:
                            receiving = False
                if len(msg) == 0:
                    if current_socket in users_dict.keys():
                        disconnect(users_dict[current_socket])
                else:

                    print("GOT FROM CLIENT", msg)
                    if current_socket in waiting_clients.keys():
                        # sending  the data from client to browser
                        waiting_clients[current_socket].send(msg)
                    

                    else:
                        msg = msg.decode()
                        msgSplit = msg.split()
                        address = msgSplit[1]

                        if address.split(':')[1].isnumeric():
                            if msg.startswith('CONNECT'):
                                browserLink, browserPort = address.split(':')
                                browserPort = int(browserPort)
                                browserIP = socket.gethostbyname(browserLink)
                                address = (browserIP, browserPort)
                                # connect to the site
                                browserSocket = socket.socket()
                                print(address)
                                browserSocket.connect((browserIP, browserPort))
                                waiting_clients[current_socket] = browserSocket
                                browsers_clients[browserSocket] = current_socket
                                msg_ret = "HTTP/1.1 200 Connection established\r\n\r\n"
                                sendMsg(users_dict[current_socket], msg_ret)
                                m

代理能够在 CONNECT 之后建立连接并通知客户端，但是在我发送到浏览器之后，我从后台运行的函数中获取了数据：

def browserCom():
    while True:
        try:
            rlist, wlist, xlist = select.select(list(browsers_clients.keys()), [], [], 0.3)
        except:
            pass
        else:
            for current_browser in rlist:
                # receive data from the browser
                receiving = True
                resp_msg = bytearray()
                while receiving:
                    try:
                        data = current_browser.recv(1024)
                    except Exception as e:
                        print(e)
                        del waiting_clients[browsers_clients[current_browser]]
                        current_browser.close()
                        browsers_clients[current_browser].close()
                        del browsers_clients[current_browser]

                    else:
                        resp_msg.extend(data)
                        # got the full msg
                        if len(data) < 1024:
                            receiving = False

                print("RESPONSE FROM BROWSER", resp_msg)
                # sending the msg to the client
                sendMsg(users_dict[browsers_clients[current_browser]], resp_msg)

我需要等待很长时间才能得到响应，并且大多数响应都是空的，响应大多是 bytearray(b'')，即使我收到响应，即使我将响应发送回客户端：

# sending the msg to the client
sendMsg(users_dict[browsers_clients[current_browser]],resp_msg)

使用这个

def sendMsg(address, msg):
        """
    
        :param ip: ip to send to
        :param msg: msg to send
        :return: sends the msg to the ip
        """
        if address in open_clients.keys():
            sock = open_clients[address]
            if type(msg) == str:
                msg = msg.encode()
            try:
                sock.send(msg)
            except Exception as e:
                print(e, 4)
                disconnect(address)

希望你能看懂我的代码，如果有不清楚的地方请在cmets中问我，我会尽快帮助你理解

这是我可以做的最好的事情，以在不删除关键部分的情况下保持代码最少

【问题讨论】：

我不确定您要做什么，这可能是因为不完整。您只显示您认为相关的部分，但我认为这还不够。请参阅How to create a Minimal, Reproducible Example。
我已经阅读并更新了代码，希望现在更好
从简短的代码来看，有两件事很突出。首先，您似乎假设在 CONNECT 隧道内只有从客户端到服务器的数据，然后是从服务器到客户端的数据。这个假设是错误的，数据将在 TLS 连接内多次双向发送。其次，您正在使用阻塞套接字，并假设如果您读取了所有请求的 1024 字节，那么将有更多数据可用。如果恰好有 1024 个字节要读取，则此假设是错误的 - 在这种情况下，您的代码将阻塞。
“我正在使用“选择”打开一个线程” - 选择不会打开任何线程。您在此处的所有代码都在单个线程中运行，并且一个套接字上的任何阻塞 recv 或接受都将意味着任何其他套接字也不会取得任何进展。
我之前提到的错误期望仍然存在：" 您似乎假设在 CONNECT 隧道内只有从客户端到服务器的数据，然后是从服务器到客户端的数据。这种假设是错误的，数据将在 TLS 连接内多次双向发送”。您只需等待服务器关闭连接 - 这就是导致您看到的问题的原因。相反，您需要检查客户端和服务器端的读取，然后将读取的数据从每一端传输到另一端。

标签： python sockets https proxy

【解决方案1】：

我的错误是我不明白在隧道浏览器和客户端交换消息时，向所有浏览器添加另一个选择有助于我检查来自所有浏览器的所有数据，现在它可以工作了。

上面的代码已经更新并且可以工作了

基本上我添加的是：

def browserCom():
while True:
    try:
        rlist, wlist, xlist = select.select(list(browsers_clients.keys()), [], [], 0.3)
    except:
        pass
    else:
        for current_browser in rlist:
            # receive data from the browser
            receiving = True
            resp_msg = bytearray()
            while receiving:
                try:
                    data = current_browser.recv(1024)
                except Exception as e:
                    print(e)
                    del waiting_clients[browsers_clients[current_browser]]
                    current_browser.close()
                    browsers_clients[current_browser].close()
                    del browsers_clients[current_browser]

                else:
                    resp_msg.extend(data)
                    # got the full msg
                    if len(data) < 1024:
                        receiving = False
            # disconnecting browser
            if resp_msg == bytearray(b''):
                del waiting_clients[browsers_clients[current_browser]]
                current_browser.close()
                browsers_clients[current_browser].close()
                del browsers_clients[current_browser]

            print("RESPONSE FROM BROWSER", resp_msg)
            # sending the msg to the client
            if current_browser in browsers_clients and browsers_clients[current_browser] in users_dict:
                sendMsg(users_dict[browsers_clients[current_browser]], resp_msg)

【讨论】：