如何使用 requests.post 在网站上提交表单？答案

【问题标题】：How can I use requests.post to submit a form on a website?如何使用 requests.post 在网站上提交表单？
【发布时间】：2021-04-16 22:55:19
【问题描述】：

我正在尝试使用请求为https://immoweb.be 上发布的每个广告提交查询表单。每当发送请求时，都会生成一个唯一的 x-xsrf-token 并在 POST 请求中附加标头。因此，每隔几个小时，由于令牌过期而发生 419 错误。我正在使用 https://curl.trillworks.com/ 为 python 脚本创建标头和有效负载。

import requests
header1 = {
'authority': 'www.immoweb.be',
'accept': 'application/json, text/plain, */*',
'x-xsrf-token': 'value here',
'x-requested-with': 'XMLHttpRequest',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.128 Safari/537.36',
'content-type': 'application/json;charset=UTF-8',
'origin': 'https://www.immoweb.be',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.immoweb.be/en/classified/apartment/for-sale/merksem/2170/9284079?searchId=607a09e4e4532',
'accept-language': 'en-US,en;q=0.9',
'cookie': 'Cookie data here',
}
data1 = '{"firstName":"Name here","lastName":"Name here","email":"myemail@gmail.com","classifiedId":9284079,"customerIds":\[add ehre\],"phone":"","message":"I am interested in your property. Can you give me some more information in order to plan a possible visit? Thank you.","isUnloggedUserInfoRemembered":false,"sendMeACopy":false}'
response = s.post('https://www.immoweb.be/nl/email/request', headers=header1, data=data1)  
print (response.status_code)  
print (response.cookies)

【问题讨论】：

标签： python python-3.x web-scraping python-requests

【解决方案1】：

你可以从cookie中获取新的X-XSRF-TOKEN，例如：

import requests
from urllib.parse import unquote

url = "https://www.immoweb.be"

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0"
}

json_data = {
    "classifiedId": 9234221,  # <-- change this
    "customerIds": [305309],  # <-- change this
    "email": "sample@example.com",
    "firstName": "xxx",
    "isUnloggedUserInfoRemembered": False,
    "lastName": "xxx",
    "message": "I am interested in your property. Can you give me some more information in order to plan a possible visit? Thank you.",
    "phone": "",
    "sendMeACopy": False,
}

with requests.Session() as s:
    # this is only for getting first "XSRF-TOKEN"
    s.get(url)
    token = unquote(s.cookies["XSRF-TOKEN"])

    # use this token to post request:
    headers["X-Requested-With"] = "XMLHttpRequest"
    headers["X-XSRF-TOKEN"] = token
    status = s.post(
        "https://www.immoweb.be/en/email/request",
        headers=headers,
        json=json_data,
    ).json()

    print(status)

    # new XSRF-TOKEN is here:
    token = unquote(s.cookies["XSRF-TOKEN"])
    headers["X-XSRF-TOKEN"] = token

    # repeat
    # ...

打印：

{'success': True}

【讨论】：