【发布时间】:2015-12-31 22:12:40
【问题描述】:
我完全迷路了。这是我的管道。当我运行它时,我收到一个错误提示
File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "C:\Python27\bff\bff\pipelines.py", line 42, in process_item
cursor.execute(add_Product)
File "c:\python27\lib\site-packages\mysql\connector\cursor.py", line 492, in execute
stmt = operation.encode(self._connection.python_charset)
AttributeError: 'tuple' object has no attribute 'encode'
从注释代码可以看出,我尝试了几种不同的方法。起初,我按照在示例中看到的那样进行操作,但是当我将 (item['StoreName']) 放入 VALUES 行而不是在上面将其定义为 Name = item 时,我得到了一个错误,即未定义 sadi item ['商店名称'] 我正在使用从 mqsql.org 网站安装的 mySQL.connector。在此先感谢
# -*- coding: utf-8 -*-
# Define your item pipelines here
# Don't forget to add your pipeline to the ITEM_PIPELINES setting
# See: http://doc.scrapy.org/en/latest/topics/item-pipeline.html
from __future__ import print_function
from datetime import date, datetime, timedelta
import mysql.connector
#from scrapy.extensions import DropItem
#from bff.items import ItemInfo
class mySQLPipeline(object):
def process_item(self, item, spider):
Path = item['ProdPath']
UPC = item['ProdUPC']
Model = item['ProdModel']
Desc = item['ProdDesc']
Price = item['ProdPrice']
Stock = item['InStock']
#Ships = item['Ships']
Name = item['StoreName']
cnx = mysql.connector.connect(user='*****', password='*****',
host='127.0.0.1',
port='****',
database='scrapyinfo')
cursor = cnx.cursor()
#add_Product = ("INSERT INTO walmart_products (ProdName)"
# "VALUES (%s), (Name);")
add_Product = ("INSERT INTO walmart_products, (ProdName, ProdPath, ProdUPC, ProdModel, ProdDesc, ProdPrice, InStock, Ships, StoreName)"
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)", (Name, Path, UPC, Model, Desc, Price, Stock, Name))
#item['Ships'],
#Add new product
cursor.execute(add_Product)
# Make sure data is committed to the database
cnx.commit()
cursor.close()
cnx.close()
return item
编辑。这是我的新代码
`
from __future__ import print_function
from datetime import date, datetime, timedelta
import mysql.connector
#from scrapy.extensions import DropItem
#from bff.items import ItemInfo
class mySQLPipeline(object):
def process_item(self, item, spider):
Product = item['ProdName']
Path = item['ProdPath']
UPC = item['ProdUPC']
Model = item['ProdModel']
Desc = item['ProdDesc']
Price = item['ProdPrice']
Stock = item['InStock']
#Ships = item['Ships']
Name = item['StoreName']
cnx = mysql.connector.connect(user='****', password='****',
host='127.0.0.1',
port='****',
database='****')
cursor = cnx.cursor()
# add_Product = ("INSERT INTO walmart_products (ProdName, StoreName) VALUES (%s, %s,)", Product, Name,)
# add_Product = ("INSERT INTO walmart_products, (ProdName)"
# "VALUES (%s)", (Name))
# "VALUES (%(Name)s)")
add_Product = ("INSERT INTO walmart_products "
"(ProdName, ProdPath, ProdUPC, ProdModel, ProdDesc, ProdPrice, InStock, StoreName) "
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s)")
#item['Ships'],
data_Product = (Product, Path, UPC, Model, Desc, Price, Stock, Name)
#Add new product
cursor.execute(add_Product, data_Product)
# Make sure data is committed to the database
cnx.commit()
cursor.close()
cnx.close()
return item
`
【问题讨论】:
-
字段填充的是列表还是字符串?
-
没有列表。这是我的 items.py
# -*- coding: utf-8 -*- # Define here the models for your scraped items # # See documentation in: # http://doc.scrapy.org/en/latest/topics/items.html import scrapy class ItemInfo(scrapy.Item): ProdPath = scrapy.Field() ProdUPC = scrapy.Field() ProdName = scrapy.Field() ProdModel = scrapy.Field() ProdDesc = scrapy.Field() InStock = scrapy.Field() ProdPrice = scrapy.Field() StoreName = scrapy.Field() Ships = scrapy.Field() pass的代码 -
分享您如何填充项目,而不是项目声明。
-
抱歉,我试图弄清楚如何让代码在回复中正确显示。这是我如何填充项目的示例。 ` def parse_item(self, response): sel = Selector (response) item = ItemInfo() item['ProdPath'] = sel.xpath('/html/head/meta[@property="og:url"]/@ content').extract() item['ProdUPC'] = sel.xpath('/html/head/meta[@property="og:upc"]/@content').extract()`
-
.extract()给出一个列表。
标签: python mysql scrapy scrapy-pipeline