蟒蛇请求 |构造 POST 请求体

Question

我正在尝试使用 python 请求构建下面的 POST 请求。

请求标头

Host: www.example.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:46.0) Gecko/20100101 Firefox/46.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
X-CSRFToken: LdoMdAvsJ3QeFZ79YK1ZKylohMYCEgif
X-Requested-With: XMLHttpRequest
Content-Type: multipart/form-data; boundary=---------------------------6820904081792922663698482414
Cache-Control: max-age=0
Referer: https://www.example.com/def/
Content-Length: 175
Cookie: HE_UTS_ID_CL=5f0ae4142d484a5dbc6579228be42f9a6bc68c95cdd64156af65fffb49e258d4; HE_UTS_ID_LP="/challenges/"; csrftoken=LdoMdAvsJ3QeFZ79YK1ZKylohMYCEgif; user_tz=Asia/Kolkata; _ga=GA1.2.1255646438.1464624573; lordoftherings="a5dr3g48ag2dg8s2b8r57gkil6ioip74:df99595bff3625e78f4caf1281450faf"; mp_mixpanel__c=6; mp_mixpanel__c3=4136; mp_mixpanel__c4=3974; mp_mixpanel__c5=25; hiring_challenge_anon_visited=visited; mp_4c30b8635363027dfb780d5a61785112_mixpanel=%7B%22distinct_id%22%3A%20%22155026da7bb4b-0b9cbbccc1992-3e6e034e-100200-155026da7bceb%22%2C%22url_path%22%3A%20%22%2Fchallenges%2F%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; HE_UTS_ID=c5bb2531e6f9465bb0d7115d953bdbe79c17ad924dfb467f9e76fa5c19e373b2
Connection: keep-alive

请求正文

-----------------------------6820904081792922663698482414
Content-Disposition: form-data; name="submit"

True
-----------------------------6820904081792922663698482414--

以下是我现在的代码。

import requests

url = 'https://www.example.com/adadsdas?modern=true'

headers = {'Host': 'www.abc.com',\
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:46.0) Gecko/20100101 Firefox/46.0',\
'Accept': '*/*',\
'Accept-Language': 'en-US,en;q=0.5',\
'Accept-Encoding': 'gzip, deflate, br',\
'X-CSRFToken': '6jdyxOPzlE4ui1HzNa5RRIYhGAaRtalw',\
'X-Requested-With': 'XMLHttpRequest',\
'Referer': 'https://www.example.com/def/',\
'Content-Length': '175',\
'Content-Type': 'multipart/form-data; boundary=---------------------------6820904081792922663698482414',\
'Cookie': 'HE_UTS_ID_CL=4179383969bd41a489340b5ceeb4cde3225d29cfa7734ea09a8a57b0fbd1e0cd; HE_UTS_ID_LP="/challenges/"; csrftoken=6jdyxOPzlE4ui1HzNa5RRIYhGAaRtalw; user_tz=Asia/Kolkata; _ga=GA1.2.1255646438.1464624573; lordoftherings="a5dr3g48ag2dg8s2b8r57gkil6ioip74:df99595bff3625e78f4caf1281450faf"; mp_mixpanel__c=6; mp_mixpanel__c3=4136; mp_mixpanel__c4=3974; mp_mixpanel__c5=25; hiring_challenge_anon_visited=visited; mp_4c30b8635363027dfb780d5a61785112_mixpanel=%7B%22distinct_id%22%3A%20%22155026da7bb4b-0b9cbbccc1992-3e6e034e-100200-155026da7bceb%22%2C%22url_path%22%3A%20%22%2Fchallenges%2F%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; HE_UTS_ID=efd6ef54d30f48efbc0be67b2283545daf69267d76e6403595f683a26a5adbf7',\
'Connection': 'keep-alive',\
'Cache-Control': 'max-age=0'}

requests.post(url,headers=headers,files={'submit':(None,'True')})

我在传递文件字典时遇到异常， 如上所述构造 POST 请求的正确方法是什么？ 如何构造请求体？并查看原始请求以检查正在构建的内容？

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 87, in post
    return request('post', url, data=data, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 276, in request
    prep = req.prepare()
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 224, in prepare
    p.prepare_body(self.data, self.files)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 369, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 131, in _encode_files
    body, content_type = encode_multipart_formdata(new_fields)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/filepost.py", line 73, in encode_multipart_formdata
    content_type = get_content_type(filename)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/filepost.py", line 27, in get_content_type
    return mimetypes.guess_type(filename)[0] or 'application/octet-stream'
  File "/usr/lib/python2.7/mimetypes.py", line 298, in guess_type
    return _db.guess_type(url, strict)
  File "/usr/lib/python2.7/mimetypes.py", line 114, in guess_type
    scheme, url = urllib.splittype(url)
  File "/usr/lib/python2.7/urllib.py", line 1074, in splittype
    match = _typeprog.match(url)
TypeError: expected string or buffer

Answer 1

您的错误是从使用 2013 年的古老版本的请求 1.1.0 到这个答案的 cmets，所以 pip install -U requests 将解决这个问题。

要发布您不需要硬编码任何内容的帖子，您可以创建一个会话并获取您需要的任何内容，即 csrf 令牌和请求将负责其余的工作：

import requests   


headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:46.0) Gecko/20100101 Firefox/46.0',
           'X-Requested-With': 'XMLHttpRequest', "referer": "https://www.hackerearth.com/challenges/"}

with requests.Session() as s:
    s.get("https://www.hackerearth.com")
    headers["X-CSRFToken"] = s.cookies["csrftoken"]
    r = s.post("https://www.hackerearth.com/AJAX/filter-challenges/?modern=true", headers=headers,
               files={'submit': (None, 'True')})
    print(r.json())

三个必要的东西是，X-Requested-With，因为它必须是一个 ajax 请求，referer 和 csrf 令牌。

要访问原始请求数据，您可以访问r.request 上的属性：

print(r.request.body)
print(r.request.headers)