无法将scrapy部署到scrapyd服务器答案

【问题标题】：Can't deploy scrapy to scrapyd server无法将scrapy部署到scrapyd服务器
【发布时间】：2016-07-22 16:21:58
【问题描述】：

我正在尝试部署通过Portia 创建的蜘蛛。 Portia，scrapyd - 都是最新版本。

我只是通过命令运行 scrapyd 服务器：

scrapyd

我在本地机器上得到了这个结果：

$> cd PROJECT_PATH_HERE;
$> scrapyd-deploy
Packing version 1463420166
Deploying to project "retail" in http://X.X.X.X:6800/addversion.json
Server response (200):
{"status": "error", "message": "AttributeError: 'list' object has no attribute 'iteritems'", "node_name": "vaua0048313.online-vm.com"}

在服务器端（Ubuntu 12.04.5 LTS）：

root@server:~# scrapyd
2016-05-16 17:35:42+0000 [-] Log opened.
2016-05-16 17:35:42+0000 [-] twistd 16.1.1 (/usr/bin/python 2.7.3) starting up.
2016-05-16 17:35:42+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2016-05-16 17:35:42+0000 [-] Site starting on 6800
2016-05-16 17:35:42+0000 [-] Starting factory <twisted.web.server.Site instance at 0x2f5d950>
2016-05-16 17:35:42+0000 [Launcher] Scrapyd 1.1.0 started: max_proc=12, runner='scrapyd.runner'
2016-05-16 17:36:06+0000 [_GenericHTTPChannelProtocol,0,78.111.181.96] Unhandled Error
        Traceback (most recent call last):
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 1767, in allContentReceived
            req.requestReceived(command, path, version)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 768, in requestReceived
            self.process()
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 183, in process
            self.render(resrc)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 234, in render
            body = resrc.render(self)
        --- <exception caught here> ---
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/webservice.py", line 17, in render
            return JsonResource.render(self, txrequest)
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/utils.py", line 19, in render
            r = resource.Resource.render(self, txrequest)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/resource.py", line 250, in render
            return m(request)
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/webservice.py", line 79, in render_POST
            spiders = get_spider_list(project, version=version)
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/utils.py", line 116, in get_spider_list
            raise RuntimeError(msg.splitlines()[-1])
        exceptions.RuntimeError: AttributeError: 'list' object has no attribute 'iteritems'

2016-05-16 17:36:06+0000 [-] "78.111.181.96" - - [16/May/2016:17:36:05 +0000] "POST /addversion.json HTTP/1.1" 200 135 "-" "Python-urllib/2.7"

我做错了什么？

【问题讨论】：

您能否发布您的 slybot、scrapy 和 scrapyd 版本，以便我们尝试重新创建它？
@RuairiFahy slybot==0.13.0b18, Scrapy==1.0.0, scrapyd-client==1.0.1

标签： scrapy scrapy-spider scrapyd

【解决方案1】：

我认为问题出在您的 setting.py 中。 ITEM_PIPELINES 应该由下面的 dict 格式指定，当你使用的是 1.10.0 的 scrapy 版本时，

ITEM_PIPELINES = {
    'coolscrapy.pipelines.ArticleDataBasePipeline': 5,
}

【讨论】：

【解决方案2】：

将Portia_Projects/spiders/setting.py中的ITEM_PIPELINES修改为

ITEM_PIPELINES = {'slybot.dupefilter.DupeFilterPipeline':1}

【讨论】：