【问题标题】:Error in scrapy screen scraper - Cant find what's wrong for the life of me刮屏刮板出错 - 找不到我的生活有什么问题
【发布时间】:2015-02-09 02:51:02
【问题描述】:

我不知道是什么导致了这个错误。该错误发生在 craig.py 文件的第 3 行,但我没有看到任何差异。

文件夹结构

  • 克雷格(文件夹)
    • 蜘蛛(文件夹)
      • 初始化.py
      • 初始化.pyc
      • craig.py
      • craig.pyc
    • 初始化.py
    • 初始化.pyc
    • pipelines.py
    • settings.py
    • settings.pyc
    • scrapy.cfg

项目名称:克雷格 文件名称:克雷格 间谍名称:Craig.py

Craig.py

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from craig.items import CraigslistSampleItem

class MySpider (BaseSpider):
    name = "craig"
    allowed_domain = ["craigslist.org"]
    start_urls = ["http://sfbay.craigslist.org/sfc/npo/"]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        title = hxs.select("//p")
        items = []
        for titles in titles:
            item = CraigslistSampleItem()
            item ["title"] = titles.select("a/text()").extract()
            item ["link"] = titles.select("a/@href").extract()
            items.append(item)
        return items

items.py

# -*- coding: utf-8 -*-

# Define here the models for your scraped items
#
# See documentation in:
# http://doc.scrapy.org/en/latest/topics/items.html

from scrapy.item import Item, Field


class CraigslistSampleItem(Item):
    title = Field()
    link = Field()

错误如下:

Traceback (most recent call last):
  File "C:\Python27\Scripts\scrapy-script.py", line 9, in <module>
    load_entry_point('scrapy==0.24.4', 'console_scripts', 'scrapy')()
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\cmdline.py"
, line 143, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\cmdline.py"
, line 89, in _run_print_help
    func(*a, **kw)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\cmdline.py"
, line 150, in _run_command
    cmd.run(args, opts)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\commands\cr
awl.py", line 57, in run
    crawler = self.crawler_process.create_crawler()
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\crawler.py"
, line 87, in create_crawler
    self.crawlers[name] = Crawler(self.settings)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\crawler.py"
, line 25, in __init__
    self.spiders = spman_cls.from_crawler(self)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\spidermanag
er.py", line 35, in from_crawler
    sm = cls.from_settings(crawler.settings)
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\spidermanag
er.py", line 31, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\spidermanag
er.py", line 22, in __init__
    for module in walk_modules(name):
  File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\utils\misc.
py", line 68, in walk_modules
    submod = import_module(fullpath)
  File "C:\Python27\lib\importlib\__init__.py", line 37, in import_module
    __import__(name)
  File "C:\Users\Turbo\craig\craig\spiders\craig.py", line 3, in <module>
    from craig.items import CraigslistSampleItem
ImportError: No module named items

【问题讨论】:

  • 您没有在文件夹结构中显示ıtems.py 文件的位置。这就是原因,你把它保存在一个完全随机的地方:D

标签: python python-2.7 scrapy screen-scraping scrapy-spider


【解决方案1】:

请显示 items.py 在项目结构中的位置。

你应该是这样的:

  • 克雷格(文件夹)
    • craig.py
    • 项目(文件夹)
      • __ 初始化 __.py
      • items.py

【讨论】:

  • 好的。给你,我改变了帖子:)
  • 但我没有看到“items.py”文件。
  • 为什么这个答案被接受了?解决方案是什么?
猜你喜欢
  • 2023-04-02
  • 1970-01-01
  • 1970-01-01
  • 2022-01-02
  • 2011-07-22
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2016-01-09
相关资源
最近更新 更多