在列表中附加提取的链接，但列表在打印时给出整个标签而不是链接

Question

这是我的代码

from bs4 import BeautifulSoup
import requests, lxml
import re
from urllib.parse import urljoin
from googlesearch import search
import pandas as pd

query = 'A M C College of Engineering, Bangalore'
link = []

for i in search(query, tld='co.in', start=0, stop=1):
    print(i)
    soup = BeautifulSoup(requests.get(i).text, 'lxml')
    for link in soup.select("a[href$='.pdf']"):
        if re.search(r'nirf', str(link), flags=re.IGNORECASE):
            fUrl = urljoin(i, link['href'])
            print(fUrl)
            link.append(fUrl)
print(link)
df = pd.DataFrame(link, columns=['PDF LINKS'])
print(df)

这是我运行代码后的输出：

https://www.amcgroup.edu.in/AMCEC/index.php
https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFENGG.pdf
https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFMBA.pdf
https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2019.pdf
https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2020.pdf
# Printing list with links but getting tags
<a href="image/gallery/Swami Vivekananda.pdf" target="_black">For Invitation Click here...</a>
# Dataframe where I want to store list
                      PDF LINKS
0  For Invitation Click here...

我应该得到显示在输出中的链接列表，但是在打印列表时它给了我整个标签而不是链接。我还想将我进入的所有链接推送到单行数据框中，如下所示：

       PDF LINKS
0  link1 link2 link3    #for query1
1  link1 link2          #for another query

我怎样才能做到这一点。我的代码有什么问题，为什么我得到标签而不是列表。 提前致谢。

Answer 1

为列表和for循环中的标签使用不同的变量名：

import re
import requests
import pandas as pd
from bs4 import BeautifulSoup
from urllib.parse import urljoin


query = "A M C College of Engineering, Bangalore"
all_data = []

for i in ["https://www.amcgroup.edu.in/AMCEC/index.php"]:
    soup = BeautifulSoup(requests.get(i).text, "lxml")
    for link in soup.select("a[href$='.pdf']"):  # <-- `link` is different than `all_data` here!
        if re.search(r"nirf", link["href"], flags=re.IGNORECASE):
            fUrl = urljoin(i, link["href"])
            all_data.append(fUrl)

df = pd.DataFrame(all_data, columns=["PDF LINKS"])
print(df)

打印：

                                                        PDF LINKS
0   https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFENGG.pdf
1    https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFMBA.pdf
2  https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2019.pdf
3  https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2020.pdf

---

编辑：将结果放在一行中：

import re
import requests
import pandas as pd
from bs4 import BeautifulSoup
from urllib.parse import urljoin


query = "A M C College of Engineering, Bangalore"
all_data = []

for i in ["https://www.amcgroup.edu.in/AMCEC/index.php"]:
    soup = BeautifulSoup(requests.get(i).text, "lxml")
    row = []
    for link in soup.select(
        "a[href$='.pdf']"
    ):  # <-- `link` is different than `all_data` here!
        if re.search(r"nirf", link["href"], flags=re.IGNORECASE):
            fUrl = urljoin(i, link["href"])
            row.append(fUrl)
    if row:
        all_data.append(row)

df = pd.DataFrame({"PDF LINKS": all_data})
print(df)

打印：

                                                                                                                                                                                                                                                       PDF LINKS
0  [https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFENGG.pdf, https://www.amcgroup.edu.in/AMCEC/image/Download/NIRFMBA.pdf, https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2019.pdf, https://www.amcgroup.edu.in/AMCEC/image/Download/NIRF_2020.pdf]