Python，将所有链接、标题和正文文本附加到一个数组或 json 文件中

Question

帮助，我正在从一个网站获取多个 href url 链接，并尝试将 url 的每个标题和正文附加到另一个数组中。但是，当我运行类似的东西时，我只抓住一个标题，将其他链接的所有文本放在一起。

request = requests.get(url)
somecontents = request.content
soup = BeautifulSoup(somecontents, "html.parser")
soup.prettify()
gethref = urllinks.get("href")

if gethref is not None and\
  "http" in gethref and\
  "photo" not in gethref and\
  "img" not in gethref:
    page_links = []
    tags_in_link = gethref
    page_links.append(tags_in_link)
    hrefdataset = ','.join(page_links)

for each_link in i:
    website_header_title = soup.title.string
    parse_title = re.sub('[^A-Za-z]+', ' ', website_header_title)
    time.sleep(.05)

    done = grab_web_text(each_link)

    testintry = []
    testintry.append("Website Title: " + parse_title + "," + " ")
    text = testintry.append("Body: " + done)

我想要每个链接：我怎样才能将其格式化为我所拥有的？

[{"Website Title: " "title", "Body: " "Body}, 
[{"Website Title: " "title", "Body: " "Body}, 
[{"Website Title: " "title", "Body: " "Body}, 
[{"Website Title: " "title", "Body: " "Body}]

Answer 1

您可以像这样创建一个字典列表：

def get_link_info(l):
    parse_title = re.sub('[^A-Za-z]+', ' ', website_header_title)
    done = grab_web_text(each_link)
    return (parse_title, done)

print([{t: d} for t, d in (get_link_info(i) for i in links)])

这是如何工作的？

1. for i in links 是所有链接的循环。
2. get_link_info 返回一个包含 title 和 `done
的元组
3. for t, d in (...) 是结果元组的循环
4. {t: d} for t, d in (...) 是 dict comprehension
5. 外部[] 从生成器创建一个列表。