试图从 csv 文件中提取链接到 request.get 但得到“ TypeError: 'NoneType' object is not subscriptable ”答案

【问题标题】：Trying to extract links from fetching link from csv file into request.get but getting " TypeError: 'NoneType' object is not subscriptable "试图从 csv 文件中提取链接到 request.get 但得到“ TypeError: 'NoneType' object is not subscriptable ”
【发布时间】：2019-02-28 09:12:41
【问题描述】：

代码：

from bs4 import BeautifulSoup

import requests

import csv

import pandas as pd

links = pd.read_csv('C:\\Users\\acer\\Desktop\\hindustan_pages.csv',encoding = 'latin',dtype=str)

for i in range(1,3):

      link = links.iloc[i,0]

      r = requests.get(link)

      soup = BeautifulSoup(r.text,'lxml')

      div = soup.find('div',{"id":"company_list_grid"})
      for links in div.find_all('th',{"id":"c_name"}):
          link = links.find('a')
          print("https://www.hindustanyellowpages.in/Ahmedabad" + link['href'][2:])

出现错误：

Traceback（最近一次调用最后一次）：

文件 "C:\Users\acer\AppData\Local\Programs\Python\Python37\hindustanyellowpages.py", 第 8 行，在
link = links.iloc[i,0]
TypeError: 'NoneType' 对象不可下标

请帮我解决这个问题。

【问题讨论】：

我已将 csv 文件发送给您，请检查。

标签： python-3.x beautifulsoup

【解决方案1】：

发现问题，但首先：

1) 当 pandas 读取 csv 文件时，假设您有一个标题。所以你的csv中的第1行实际上永远不会被处理，因为它被存储为列名，所以你需要添加一个参数来告诉pandas没有标题header=None。还请注意，不确定您是否意识到这一点，但索引从 0 开始。因此，您的代码实际上是从数据帧第 2 行的链接开始的。如果您希望它从第 1 行开始，您需要 range(0,2)

2) TypeError: 'NoneType' object is not subscriptable 被引发，因为 links 存储您的数据帧以进行迭代，在 for links in div.find_all('th',{"id":"c_name"}): 处被覆盖。因此，当它进入 links 数据框的第二行时，它不再是数据框，而是 div.find_all('th',{"id":"c_name"}) 标记中的元素。为了解决这个问题，我们只需将您的 links 数据框重命名为 links_df：

from bs4 import BeautifulSoup
import requests
import csv
import pandas as pd

# Renamed to links_df and added header=None
links_df = pd.read_csv('C:\\Users\\acer\\Desktop\\hindustan_pages.csv',encoding = 'latin',dtype=str, header=None)

#for i, row in links_df.iterrows():  <---- if you want it to do the whole dataframe, you can simply use .iterrows instead of for i in range()
for i in range(1,3):
    link = links_df.iloc[i,0] #<--- refers to a row in your links_df
    r = requests.get(link)
    soup = BeautifulSoup(r.text,'lxml')

    div = soup.find('div',{"id":"company_list_grid"})
    for links in div.find_all('th',{"id":"c_name"}):
        link = links.find('a')
        print("https://www.hindustanyellowpages.in/Ahmedabad" + link['href'][2:])

输出：

https://www.hindustanyellowpages.in/Ahmedabad/Hetal-Mandap-Decorators/Satellite
https://www.hindustanyellowpages.in/Ahmedabad/Radhe-Krishna-Event-Management/Bapunagar
https://www.hindustanyellowpages.in/Ahmedabad/Amiraj-Decorators/Maninagar
https://www.hindustanyellowpages.in/Ahmedabad/Hiral-Handicraft/Saijpur-Bogha
https://www.hindustanyellowpages.in/Ahmedabad/S-D-Traders/Teen-Darwaja
https://www.hindustanyellowpages.in/Ahmedabad/Agro-Net-Plast/Kathwada
https://www.hindustanyellowpages.in/Ahmedabad/Shree-Krishna-Suppliers/Naroda
https://www.hindustanyellowpages.in/Ahmedabad/Bulakhidas-Vitthaldas/Panchkuva
https://www.hindustanyellowpages.in/Ahmedabad/Nagindas-AND-Sons-(Patva)/Relief-Road
https://www.hindustanyellowpages.in/Ahmedabad/New-Rahul-Electricals-OR-Dhruv-Light-Decoration-AND-Sound-System/Subhash-Bridge
https://www.hindustanyellowpages.in/Ahmedabad/Bhairavi-Craft/Thaltej
https://www.hindustanyellowpages.in/Ahmedabad/Saath-Sangath-Party-Plot/Rakanpur
https://www.hindustanyellowpages.in/Ahmedabad/Poonam-Light-Decoration-And-Electricals/Naroda
https://www.hindustanyellowpages.in/Ahmedabad/Muku-Enterprise/Isanpur
https://www.hindustanyellowpages.in/Ahmedabad/Malaviya-Decoration-Service/Nikol
https://www.hindustanyellowpages.in/Ahmedabad/Hariprabha-Enterprises/Paldi
https://www.hindustanyellowpages.in/Ahmedabad/Festo-Craft/Thaltej
https://www.hindustanyellowpages.in/Ahmedabad/Jay-Jognimata-Decoration/Kubernagar
https://www.hindustanyellowpages.in/Ahmedabad/Maruti-Decorators/Ranip
https://www.hindustanyellowpages.in/Ahmedabad/Krishna-Light-and-Decoration/Gandhi-Road
https://www.hindustanyellowpages.in/Ahmedabad/Shree-Ambika-Light-Decoration/Ghatlodiya
https://www.hindustanyellowpages.in/Ahmedabad/R-S-Power-House/Gandhi-Road
https://www.hindustanyellowpages.in/Ahmedabad/New-Amit-Stores/Paldi
https://www.hindustanyellowpages.in/Ahmedabad/Gayatri-Cetera-s-and-Decoration/Ranip
https://www.hindustanyellowpages.in/Ahmedabad/Poonam-lights-and-decoration/Navrangpura
https://www.hindustanyellowpages.in/Ahmedabad/Shree-Ambica-Engg-Works/Naroda
https://www.hindustanyellowpages.in/Ahmedabad/Vedant-Industries/Vatva-Gidc
https://www.hindustanyellowpages.in/Ahmedabad/Honest-Traders/Narol
https://www.hindustanyellowpages.in/Ahmedabad/Sai-Samarth-Industries/Nana-Chiloda
https://www.hindustanyellowpages.in/Ahmedabad/R-N-Industries/Odhav
https://www.hindustanyellowpages.in/Ahmedabad/Jay-Ambe-Enterprise/S-G-Road
https://www.hindustanyellowpages.in/Ahmedabad/Satyam-Enterprise/Narol
https://www.hindustanyellowpages.in/Ahmedabad/Maniar-And-Co/Rakhial
https://www.hindustanyellowpages.in/Ahmedabad/Shiv-Shakti-Plastic-Industries/Amraivadi
https://www.hindustanyellowpages.in/Ahmedabad/Dinesh-Engineering-Works/Kathwada
https://www.hindustanyellowpages.in/Ahmedabad/MARUTI-ENGINEERS-OR-MARUTI-SAND-BLASTING/Odhav
https://www.hindustanyellowpages.in/Ahmedabad/Cranetech-Equipments/Vatva
https://www.hindustanyellowpages.in/Ahmedabad/Suyog-Engineering-AND-Fabricators/Chhatral-GIDC
https://www.hindustanyellowpages.in/Ahmedabad/Sanghvii-Conveyors/Thaltej
https://www.hindustanyellowpages.in/Ahmedabad/Siddh-Kripa-Steel-Fab/Kathwada
https://www.hindustanyellowpages.in/Ahmedabad/Ashirvad-Industries/Vatva-Gidc
https://www.hindustanyellowpages.in/Ahmedabad/G-I-Lokhandwala/Chandola
https://www.hindustanyellowpages.in/Ahmedabad/M-P-Electrical-Engineering--OR--M-P-Crane/Naroda-Gidc
https://www.hindustanyellowpages.in/Ahmedabad/Krishna-Enterprise/Geeta-Mandir
https://www.hindustanyellowpages.in/Ahmedabad/Nmtg-Mechtrans-Techniques-Pvt-Ltd/Naroda-Gidc
https://www.hindustanyellowpages.in/Ahmedabad/Shree-Balaad-Handling-Works/Odhav
https://www.hindustanyellowpages.in/Ahmedabad/DLM-Enterprise/Vastral
https://www.hindustanyellowpages.in/Ahmedabad/Alfa-Engineers--OR--Everest-Sanitary/Amaraivadi
https://www.hindustanyellowpages.in/Ahmedabad/Parv-Engineering-Equipments/Vatva
https://www.hindustanyellowpages.in/Ahmedabad/Vikrant-Equipments/Kathwada

补充：

使用：

for i, row in links_df.head(10).iterrows():
    link = links_df.iloc[i,0]

或

for i, row in links_df.iloc[:10].iterrows():
    link = links_df.iloc[i,0]

【讨论】：

谢谢。非常好心给我解释一下。正如你所说，我可以这样做“for i, row in links_df.iterrows():”所以这里我们必须在 link_df.iterrows(1,10) 或其他内容中给出 1 到 10 的范围。
不，没有范围。只需 links_df.iterrows() 将完成整个数据框，所有行。如果你只想要前 10 行，你可以做 link_df.head(10).iterrows() 或 link_df.iloc[:10].iterrows 也记住索引从 0 开始。所以如果你做了 links_iloc[1:10].iterrows () 这将从第 2 行开始。要开始第 1 行，您需要 [0:10] 或只是 [:10]
并且不需要 for I in range 部分。只需将其替换为 links_df.iterrows() 中的 for i, 行：
我试过这个“for i, row in links_df.iterrows():”，但是我应该使用哪个元素来放入 requests.get() 。我还是行？
尝试“link_df.head(10).iterrows() OR link_df.iloc[:10]”时出错

【解决方案2】：

试试这个。

link = df.link.iloc[i]

如果我没弄错你的 df 是链接，那么

link = links.link.iloc[i]

如果这有帮助，请告诉我

【讨论】：