【发布时间】：2019-08-20 19:33:07
【问题描述】：
我很幸运遇到了一个问题，经过几天的研究，我来到这里寻求帮助。我有一个代码，传说中说，曾经可以工作。它从 Bing Translator 抓取输出。 现在人们无法让它发挥作用。我以前从未接触过 Web Scraping 和 Python，尽管我在过去的 5 天里都在学习使用 Python 进行 Web Scraping，但我仍然无法确定问题所在。
嗯，网站上有一些更新。我已经尝试在标题帖子等上更新它。但它仍然无法正常工作，并返回相同的错误。
import re
import json
import os
import argparse
from argparse import RawTextHelpFormatter
import getpass
import sys
import requests
from bs4 import UnicodeDammit
def bing_translator(text, from_, to_, proxy, user, sp):
url = 'https://www.bing.com/ttranslate?&category=&IG=27062B7B735B438C891A574ACC3E57DE&IID=translator.5034.1'
if type(user) != type(None):
user_auth = user
if type(sp) != type(None):
user_auth = user_auth + ":" + sp
#user_auth = "fg824gk:Abc.123456"
if type(proxy) != type(None):
if type(user) != type(None):
proxy_Dict = {"http":"http://"+user_auth+"@"+proxy+":8080","https":"http://"+user_auth+"@"+proxy+":8443"}
else:
proxy_Dict = {"http":"http://"+proxy+":8080","https":"http://"+proxy+":8443"}
post_header = {}
post_header['Host'] = 'www.bing.com'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0'
post_header['Accept'] = '*/*'
post_header['Accept-Language'] = 'en-US,en;q=0.5'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Referer'] = 'https://www.bing.com/'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Connection'] = 'keep-alive'
data_payload = {'text' : text, 'from' : from_, 'to' : to_}
parameters_payload = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}
if type(proxy) != type(None):
page = requests.post(url, headers = post_header, data=data_payload, proxies = proxy_Dict) #, params = payload_paramters)
else:
page = requests.post(url, headers = post_header, data=data_payload)
try:
j = json.loads(page.content[page.content.find('{'.encode("utf8")):page.content.find('}'.encode("utf8"))+1])
except ValueError:
print ('\n\n')
print (data_payload)
print ('\n\n')
print ('\n\n')
print (len(page.content))
print (page.content)
for i in page.content:
print (ord(i))
print ('\n\n')
print ('\n\n')
sys.exit(1)
return j
当我运行bing("Platypus","en","es",None,None,None) 时，预期的输出将是“Ornitorrinco”，但我得到了那个错误：
{'text': 'Platypus', 'from': 'en', 'to': 'es'}
88955
Traceback (most recent call last):
File "C:/Users/NikolasP/Desktop/adsdsadsaadsadsa.py", line 48, in bing_translator
j = json.loads(page.content[page.content.find('{'.encode("utf8")):page.content.find('}'.encode("utf8"))+1])
File "C:\Users\NikolasP\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\NikolasP\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\NikolasP\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
bing_translator("Platypus","en","es",None,None,None)
File "C:/Users/NikolasP/Desktop/adsdsadsaadsadsa.py", line 57, in bing_translator
print (ord(i))
TypeError: ord() expected string of length 1, but int found
【问题讨论】：
-
出现错误提示“json.decoder.JSONDecodeError：期望用双引号括起来的属性名称：第 1 行第 2 列（字符 1）”，代码为
data_payload = {'text' : text, 'from' : from_, 'to' : to_}。您是否尝试将其更改为data_payload = {"text" : text, "from" : from_, "to" : to_}？另外，我不认为是真正的 MVCE。您应该能够更多地隔离错误（例如，您已标记此网络抓取，但错误似乎不是由抓取本身引起的，而是与您对抓取的数据所做的事情有关。） -
进行调试，首先打印您得到的原始响应，然后再进行任何操作，以便您（和我们）知道您在处理什么
-
bing 为其翻译 API 提供免费套餐（每月 azure.microsoft.com/en-us/pricing/details/cognitive-services/… 更容易
标签： python json python-3.x web-scraping