【发布时间】:2020-07-21 03:35:33
【问题描述】:
目前我正在使用以下代码为页面上的所有鞋子抓取https://www.nike.com/w/mens-shoes-nik1zy7ok:
import requests
import json
# I used a placeholder for the anchor parameter
uri = 'https://api.nike.com/cic/browse/v1?queryid=products&country=us&endpoint=product_feed/rollup_threads/v2?filter=marketplace(US)%26filter=language(en)%26filter=employeePrice(true)%26filter=attributeIds(0f64ecc7-d624-4e91-b171-b83a03dd8550%2C16633190-45e5-4830-a068-232ac7aea82c)%26anchor={}%26consumerChannelId=d9a5bc42-4b9c-4976-858a-f159cf99c647%26count=60'
# collect all products
store = []
with requests.Session() as session:
found_all_products = False
anchor = 0
while not found_all_products:
result = session.get(uri.format(anchor)).json()
products = result['data']['products']['products']
store += products
if len(products) < 60:
found_all_products = True
else:
anchor += 24
# filter by cloudProductId to get a dictionary with unique products
cloudProductIds = set()
unique_products = []
for product in store:
if not product['cloudProductId'] in cloudProductIds:
cloudProductIds.add(product['cloudProductId'])
unique_products.append(product)
我如何编写同样的 api 请求来检索本网站的男鞋或女鞋页面上的女鞋:https://www.nike.com/w/womens-shoes-5e1x6zy7ok?我需要更改哪个参数?
【问题讨论】:
-
我运行了你的脚本,它提供了男性和女性
-
@bigbounty 检查我的新编辑,谢谢
标签: python python-3.x api web-scraping python-requests