尽管在 scrapy 框架中传递了正确的路径值，但 response.xpath 或 response.css 没有返回任何结果

Question

我正在尝试从https://www.moneycontrol.com/markets/indian-indices/ 中抓取一些数据。但是，我坚持以下步骤，尽管我提供了正确的 xpath 值，但它不会返回任何结果。任何帮助表示赞赏。

countries = response.xpath('//*[@id="indices_stocks"]/div[2]/div/div/div/div[2]/table/tbody[2]/tr/td[1]/p/a/text()').getall()

countries = response.css('#indices_stocks > div.indices > div > div > div > div.pinned > table > tbody:nth-child(2) > tr > td:nth-child(1) > p > a::text').getall()

结果：

scraping_result_using_xpath_and_css selectors

Answer 1

您的定位器错误。
试试这个：

countries = response.xpath('//table/tbody[2]/tr/td[1]/p/a')

Answer 2

浏览器将 tbody 标签添加到表中，而 scrapy 没有。所以通常最好不要将它添加到你的 xpath 中。

xpath 路径错误，可以写得更短：

In [4]: response.xpath('//table[@class="responsive"]//a/text()').getall()
Out[4]: 
['Asian Paints',
 'Axis Bank',
 'Bajaj Auto',
 'Bajaj Finance',
 'Bajaj Finserv',
 'Bharti Airtel',
 'Dr Reddys Labs',
 'HCL Tech',
 'HDFC',
 'HDFC Bank',
 'HUL',
 'ICICI Bank',
 'IndusInd Bank',
 'Infosys',
 'ITC',
 'Kotak Mahindra',
 'Larsen',
 'M&M',
 'Maruti Suzuki',
 'Nestle',
 'NTPC',
 'Power Grid Corp',
 'Reliance',
 'SBI',
 'Sun Pharma',
 'Tata Steel',
 'TCS',
 'Tech Mahindra',
 'Titan Company',
 'UltraTechCement',
 'SENSEX',
 'S&P BSE Smallcap',
 'S&P BSE Midcap',
 'S&P BSE SmallCap Select Index',
 'S&P BSE MidCap Select Index',
 'S&P BSE LargeCap',
 'S&P BSE AllCap',
 'S&P BSE 100',
 'S&P BSE 200',
 'S&P BSE 500',
 'S&P BSE BANKEX',
 'S&P BSE Auto',
 'S&P BSE Basic Materials',
 'S&P BSE Capital Goods',
 'S&P BSE Consumer Discretionary Goods & Services',
 'S&P BSE Consumer Durables',
 'S&P BSE Energy',
 'S&P BSE Finance',
 'S&P BSE FMCG',
 'S&P BSE Healthcare',
 'S&P BSE India Mfg',
 'S&P BSE Industrials',
 'S&P BSE IPO',
 'S&P BSE IT',
 'S&P BSE Metals',
 'S&P BSE Oil and Gas',
 'S&P BSE Power',
 'S&P BSE PSU',
 'S&P BSE Realty',
 'S&P BSE TECk',
 'S&P BSE Telecom',
 'S&P BSE Utilities',
 'S&P BSE Dividend Stability Index',
 'S&P BSE Enhanced Value Index',
 'S&P BSE Low Volatility Index',
 'S&P BSE Momentum Index',
 'S&P BSE Quality Index',
 'S&P BSE SENSEX 50 Index',
 'S&P BSE 150 MidCap',
 'S&P BSE 250 SmallCap',
 'S&P BSE 250 LargeMid',
 'S&P BSE 400 MidSmall',
 'NIFTY 50',
 'NIFTY NEXT 50',
 'NIFTY Midcap 100',
 'NIFTY MIDCAP 50',
 'NIFTY 100',
 'NIFTY 500',
 'NIFTY AUTO',
 'NIFTY BANK',
 'NIFTY ENERGY',
 'NIFTY FMCG',
 'NIFTY INFRA',
 'NIFTY IT',
 'NIFTY MEDIA',
 'NIFTY METAL',
 'NIFTY MNC',
 'NIFTY PHARMA',
 'NIFTY PSE',
 'NIFTY PSU BANK',
 'NIFTY REALTY',
 'NIFTY SERV SECTOR',
 'INDIA VIX',
 'SX40',
 'NIFTY GROWSECT 15',
 'NIFTY50 VALUE 20',
 'NIFTY DIV OPPS 50',
 'NIFTY MID LIQ 15',
 'NIFTY100 QUALITY 30',
 'NIFTY Smallcap 100',
 'NIFTY COMMODITIES',
 'NIFTY CONSUMPTION',
 'NIFTY FIN SERVICE',
 'NIFTY PVT BANK',
 'NIFTY MIDCAP 150',
 'NIFTY MIDSML 400',
 'NIFTY Smallcap 250',
 'NIFTY Smallcap 50',
 'SENSEX',
 'S&P BSE Auto',
 'S&P BSE Metals',
 'S&P BSE Oil and Gas',
 'NIFTY 50',
 'NIFTY NEXT 50',
 'NIFTY Midcap 100',
 'NIFTY 100',
 'NIFTY BANK',
 'NIFTY FMCG',
 'NIFTY INFRA',
 'NIFTY IT',
 'NIFTY PHARMA']