【问题标题】:Scrapyd Deploy Error: EOFError: EOF when reading a lineScrapyd部署错误:EOFError:读取一行时出现EOF
【发布时间】:2017-11-09 12:59:09
【问题描述】:

我有 2 台电脑,PC A 没有错误,爬虫部署成功,但在 PC B 上,错误发生了。

我的 Scrapyd 服务器正在运行,但是当我尝试部署我的爬虫时,出现了这些错误。

{"status": "error", "message":Traceback (most recent call last):\\n File \"/usr/lib/python2.7/runpy.py\", line 162, in _run_module_as_main\\n \"__main__\", fname, loader, pkg_name)\\n File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\\n exec code in run_globals\\n File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 40, in <module>\\n main()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 37, in main\\n execute()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py\", line 148, in execute\\n cmd.crawler_process = CrawlerProcess(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 243, in __init__\\n super(CrawlerProcess, self).__init__(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 134, in __init__\\n self.spider_loader = _get_spider_loader(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py\", line 330, in _get_spider_loader\\n return loader_cls.from_settings(settings.frozencopy())\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 61, in from_settings\\n return cls(settings)\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 25, in __init__\\n self._load_all_spiders()\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py\", line 47, in _load_all_spiders\\n for module in walk_modules(name):\\n File \"/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py\", line 71, in walk_modules\\n submod = import_module(fullpath)\\n File \"/usr/lib/python2.7/importlib/__init__.py\", line 37, in import_module\\n __import__(name)\\n File \"spiderman/spiders/scraper.py\", line 16, in <module>\\n mail = input('Email : ')\\nEOFError: EOF when reading a line\\n", "node_name": "MY PC"}

【问题讨论】:

  • 请格式化错误跟踪并提供蜘蛛的源代码(即scraper.py根据消息)。
  • 您在文件 spiderman/spiders/scraper.py 的第 16 行有错误

标签: python ubuntu scrapy web-crawler scrapyd


【解决方案1】:

你需要安装下一个包:

  • apt-get 更新
  • apt-get 安装 gcc
  • apt-get install python-dev
  • pip install scrapyd
  • pip install scrapyd-client
  • pip install beautifulsoup4
  • 点安装日期解析器

然后尝试再次部署

【讨论】:

    猜你喜欢
    • 2021-07-30
    • 1970-01-01
    • 2021-08-30
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2013-07-19
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多