使用python从xhr请求中获取数据答案

【问题标题】：Getting data from xhr request using python使用python从xhr请求中获取数据
【发布时间】：2020-05-17 04:48:38
【问题描述】：

我正在尝试在https://www.jiocinema.com/search/avengers 网站上获取电影和连续剧我已经使用 selenium 提取了电影，但我了解了 xhr 请求。我是这个概念的新手，不知道我是否可以使用 api？

API 链接为：https://prod.media.jio.com/apis/common/v3.1/search/search

xhr 响应看起来像

有什么方法可以从上面的 xhr 响应中获取数据吗？

【问题讨论】：

标签： python api xmlhttprequest

【解决方案1】：

您可以使用 requests 库来发出类似这样的发布请求...

import requests

headers = {'User-Agent':'Some user agent'}
data = requests.post('https://prod.media.jio.com/apis/common/v3.1/search/search',headers=headers).text

您可能需要标头来发出请求...

【讨论】：

【解决方案2】：

您实际上不需要硒。您在此处调用 REST-API。

只需执行以下操作：

import requests
import traceback

def searchApi(query):
    endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
    data = {
        "q": query
    }
    try:
        response = requests.post(endpoint, data=data)
        if(response.status_code == 200):
            for msg in response:
                print(msg)
    except Exception:
        print(traceback.format_exc())

用法：

searchApi("avengers")

原始输出：

{
    "code": 200,
    "message": "success",
    "data": {
        "items": [
            {
                "name": "avengers grimm",
                "type": "Movies"
            },
            {
                "name":"avengers  endgame   official trailer  hindi ",
                "type":"Videos"
            },
            {
                "name":"avengers  endgame   official trailer",
                "type":"Videos"
            },
            {
                "name":"avengers endgame   special look",
                "type":"Videos"
            }
            .... continues
        ]
    }
}

或者，如果您想直接访问数据响应。

import json

def searchApi(query):
    endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
    data = {
        "q": query
    }
    try:
        response = requests.post(endpoint, data=data)
        if(response.status_code == 200):
            response = response.json()
            for msg in response["data"]["items"]:
                print("name: ", msg["name"], "type: ", msg["type"])
    except Exception:
        print(traceback.format_exc())

格式化输出msg["name"]和msg["type"]：

name:  avengers grimm type:  Movies
name:  avengers  endgame   official trailer type:  Videos
name:  avengers endgame   special look type:  Videos
name:  avengers  endgame   official trailer  hindi  type:  Videos
name:  the avengers  earth s mightiest heroes type:  TV Shows
name:  marvel's avengers  age of ultron type:  Movies
name:  marvel's avengers assemble type:  TV Shows
name:  marvel's avengers  age of ultron   official trailer  hindi  type:  Videos
name:  marvel's avengers  age of ultron   official trailer type:  Videos
name:  marvel's the avengers type:  Movies
name:  marvel's the avengers   official trailer type:  Videos
name:  marvel's the avengers official trailer   hindi type:  Videos
name:  making of south indian avengers type:  Videos

【讨论】：