【发布时间】:2018-12-07 20:04:30
【问题描述】:
我尝试过这个程序,因为我正在从亚马逊抓取数据,但是这个程序给了我错误。而不是 get_text 我也尝试了 extract() 并且只有 strip () 他们都给出了属性错误。现在请帮助我该怎么办?
import urllib.request
from bs4 import BeautifulSoup
import pymysql.cursors
a = input ('enter the item to be searched :')
a = a.replace(" ","")
html = urllib.request.urlopen("https://www.amazon.in/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords="+a)
bsObj = BeautifulSoup(html,'lxml')
recordList = bsObj.findAll('a', class_='a-link-normal a-text-normal')
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='shopping',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
for record in recordList:
name = record.find("h2", {"class": "a-size-small a-color-base s-inline s-access-title a-text-normal", }).get_text().strip()
sale_price = record.find("span", {"class": "currencyINR"}).get_text().strip()
category = record.find("span", {"class": "a-color-base a-text-bold"}).get_text().strip()
sql = "INSERT INTO `amazon` (`name`, `sale_price`, `category`) VALUES (%s, %s, %s)"
cursor.execute(sql, (name, sale_price, category))
connection.commit()
finally:
connection.close()
【问题讨论】:
-
这只是意味着你调用
record.find的结果是None。
标签: python python-3.x web-scraping urllib