【问题标题】:Python Requests AJAX Response Different from Browser Due to Cookie Handling由于 Cookie 处理,Python 请求 AJAX 响应与浏览器不同
【发布时间】:2020-04-10 21:51:03
【问题描述】:

我的要求:

# python 3.7.3
import requests
from requests import Session

session = Session()
session.head('https://www.basspro.com/shop/en/blazer-brass-handgun-ammo')
cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
response = session.post(
    url='https://www.basspro.com/shop/BPSGetInventoryStatusByIDView',
    data={
        'productId': '3074457345616736172',
        'itemId': '3074457345616736949',
        'isGunFlag': 'false',
    },
    cookies=cookies,
    headers={
        'accept': '*/*',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'content-length': '72',
        'content-type': 'application/x-www-form-urlencoded',
        'origin': 'https://www.basspro.com',
        'referer': 'https://www.basspro.com/shop/en/blazer-brass-handgun-ammo',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-origin',
        'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.92 Safari/537.36 Vivaldi/2.9.1705.38',
        'x-requested-with': 'XMLHttpRequest',

    },
)

print(response.text)

输出:

<input type="hidden" class="relativeToAbsolute" value="true" />

/*
{
"onlineInventory": {
        "status": "Status Not Available",
        "image": "widget_product_info/outofstock_icon.svg",
        "altText": "Status Not Available",
        "isDropShip": false,
        "availableDate":""
},
"inStoreInventory": {
        "stores": [],
        "checkStoreText": "Check Store Availability",
        "isInStoreInventory": true,
        "isPickupInventory": false
}
}
*/

通过浏览器检查和运行相同的 AJAX 请求时的输出:

/*
{
"onlineInventory": {
    "status": "Backordered",
    "image": "widget_product_info/backordered_icon.svg",
    "altText": "Backordered",
    "isDropShip": false,    
    "quantity": 0,
    "availableDate":"May 1-8"
},
"inStoreInventory": {
    "stores": [{
                id: '715839555',
                name: '83',
                gunRestricted: 'false',
                dsName: 'TX - Round Rock',
                status: 'Unavailable',
                statusText: 'Out of Stock',
                image: 'widget_product_info/outofstock_icon.svg',
                altText: 'Out of Stock',
                availableDate: '',
                availableQuantity: '',
                availableQuantityDisplay: 'false',
                cityState: 'Round Rock, TX',
                ISPavailableDate: '',
                ISPavailableQuantity: '',
                pickupTime: 'by 2:00pm',
                offerISPOnBPS: 'Yes',
                offerISPOnCAB: 'No'}],
    "checkStoreText": "Change Store",
    "isInStoreInventory": true,
    "isPickupInventory": true       
}
}
*/

我也尝试过这样分配 cookie:

url = "https://www.basspro.com/shop/en/blazer-brass-handgun-ammo"
r = requests.get(url)
cookies = r.cookies
# fails to pass the right cookie

如果我改为从https://www.basspro.com/shop/en/blazer-brass-handgun-ammo 处检查的 GET 请求中逐字复制 cookie 并将其放入 POST 标头中,它可以工作。如何让 cookie 以编程方式正常工作?


编辑:

这是我尝试将 Session() 用于 cookie:

# python 3.7.3
import requests
from requests import Session

session = Session()
session.get("https://www.basspro.com/shop/en/blazer-brass-handgun-ammo")
# session.head('https://www.basspro.com/shop/en/blazer-brass-handgun-ammo')
response = session.post(
    url='https://www.basspro.com/shop/BPSGetInventoryStatusByIDView',
    data={
        'productId': '3074457345616736172',
        'itemId': '3074457345616736949',
        'isGunFlag': 'false',
    },
    headers={
        'accept': '*/*',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'content-length': '72',
        'content-type': 'application/x-www-form-urlencoded',
        'origin': 'https://www.basspro.com',
        'referer': 'https://www.basspro.com/shop/en/blazer-brass-handgun-ammo',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-origin',
        'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.92 Safari/537.36 Vivaldi/2.9.1705.38',
        'x-requested-with': 'XMLHttpRequest',

    },
)

print(response.text)

我得到和以前一样的结果(“status”:“Status Not Available”等)

这是我对第二种解决方案的尝试:

# python 3.7.3
import requests
from requests import Session

url = "https://www.basspro.com/shop/en/blazer-brass-handgun-ammo"
r = requests.get(url)
cookies = r.cookies  # the type is RequestsCookieJar
response = requests.post(
    url='https://www.basspro.com/shop/BPSGetInventoryStatusByIDView',
    data={
        'productId': '3074457345616736172',
        'itemId': '3074457345616736949',
        'isGunFlag': 'false',
    },
    cookies=cookies,
    headers={
        'accept': '*/*',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'content-length': '72',

        'content-type': 'application/x-www-form-urlencoded',
        'origin': 'https://www.basspro.com',
        'referer': 'https://www.basspro.com/shop/en/blazer-brass-handgun-ammo',
        'sec-fetch-mode': 'cors',
        'sec-fetch-site': 'same-origin',
        'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.92 Safari/537.36 Vivaldi/2.9.1705.38',
        'x-requested-with': 'XMLHttpRequest',

    },
)

print(response.text)

再次,我得到与以前相同的结果。我做错了什么?

【问题讨论】:

  • 网站卖武器?:)
  • 是的。它们本质上没有善恶之分……这就是你如何使用它们。

标签: python ajax python-requests


【解决方案1】:

你可以这样试试

session = Session()
session.get("https://www.basspro.com/shop/en/blazer-brass-handgun-ammo")

那么下面的所有调用都用

session.xxx

不要在里面使用cookies参数

我测试过的另一种方法,

cookies = r.cookies  # the type is RequestsCookieJar
requests.post(.... cookies=cookies...)

最后,我测试了这个作品: 请仔细比较


from requests import Session

session = Session()
agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36'
r1 = session.get("https://www.basspro.com/shop/en/blazer-brass-handgun-ammo",headers={'user-agent': agent})
response = session.post(
    url='https://www.basspro.com/shop/BPSGetOnlineInventoryStatusByIDView',
    data={
        'productId': '3074457345616736172',
        'catalogId': '3074457345616676768',
        'storeId': '715838534',
        'langId':-1
    },
    headers={
        'user-agent': agent,
        'x-requested-with': 'XMLHttpRequest',
    },
    cookies=r1.cookies
)

print(response.text)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2011-09-07
    • 2023-03-24
    • 1970-01-01
    • 2020-01-01
    • 2016-01-11
    • 2018-11-16
    相关资源
    最近更新 更多