【发布时间】:2021-12-18 21:43:48
【问题描述】:
我不明白为什么我的蜘蛛无法运行。我单独测试了css选择器,所以我认为不是解析方法。
回溯消息: ReactorNotRestartable:
class espn_spider(scrapy.Spider):
name = "fsu2021_spider"
def start_requests(self):
urls = "https://www.espn.com/college-football/team/_/id/52"
for url in urls:
yield scrapy.Request(url = url, callback = self.parse_front)
def parse(self, response):
schedule_link = response.css('div.global-nav-container li > a::attr(href)')
process = CrawlerProcess()
process.crawl(espn_spider)
process.start()
【问题讨论】:
标签: python scrapy web-crawler