从画廊网站收集描述的基本刮板

Question

链接：https://vsco.co/chrysc/gallery

是否可以创建一个爬虫进入该链接并点击每张图片，将描述写入单个文本文件？

例如：

文本文件看起来像：

Circle Loop #vsco #kusama
#vsco #kusama ✨
#vsco White has eyes?
etc...

我在想“BeaustufilSoup4”或“Selenium”可以工作吗？这会是一个困难的脚本，我应该手动完成这个过程吗？

Answer 1

此脚本会将所有描述打印到屏幕和文件中：

import re
import json
import requests

url = 'https://vsco.co/chrysc/gallery'

html_data = requests.get(url).text

data = json.loads(re.search(r'window\.__PRELOADED_STATE__ = (\{.*\})', html_data).group(1))

# uncomment this to see all data:
# print(json.dumps(data, indent=4))

# print description to the screen:
for img in data['entities']['images'].values():
    print(img['description'])

# print description to the file:
with open('output.txt', 'w') as f_out:
    for img in data['entities']['images'].values():
        print(img['description'], file=f_out)

打印：

Circle Loop #vsco #kusama
#vsco #kusama ✨
#vsco White has eyes?
missing nights like these #vsco
#vsco :)
#vsco
#vsco Water//Alberta
who she is, is a result of who she was 
#vsco #chrystene peace out
#vsco #chrystene DO NOT DISTURB
#vsco #chrystene Beautiful Morning
#vsco #chrystene Paradise
#vsco Wandering Fairy Cat ✨?
#vsco close your eyes so you’re less embarrassed ??? 

并将相同的内容写入名为output.txt的文件中。