【发布时间】:2021-09-04 00:22:42
【问题描述】:
由于表格数据是由 JavaScript 动态生成的,这就是我使用 api 调用 url 的原因 但抛出上述警告。感谢任何人的帮助。
我的代码:
import requests
import pandas as pd
import json
body = { 'tipoEmpresa': '0'}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
'x-dtpc': '33$511511524_409h2vHHVRBIAIGILPJNCRGRCECUBIACWCBUEE-0e37',
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/json'
}
def main(url):
with requests.Session() as req:
req.headers.update(headers)
r = req.post(url, data=json.dumps(body), headers =headers, verify = False)
resp = r.json()['d']
#df = pd.DataFrame(resp)
#print(df)
main('https://www.rad.cvm.gov.br/ENET/frmConsultaExternaCVM.aspx/PopulaComboEmpresas')
【问题讨论】:
标签: python pandas web-scraping post python-requests