将 Dict 附加到每个请求的嵌套列表

Question

我目前正在抓取一个 XML API 响应。我希望为每个请求收集一条信息，并在每次找到这条数据时创建一个字典。每个请求可以有多个 ID。因此，一个响应可以有 2 个 ID，而下一个响应可能有 3 个 ID。例如，假设第一个响应有 2 个 ID。当第二个请求完成时，我将这些数据存储在一个列表中，另外 3 个 ID 也存储在同一个列表下。

import requests
import pandas as pd
from pandas import DataFrame
from bs4 import BeautifulSoup
import datetime as datetime
import json
import time


trackingDomain = ''
domain = ''
aIDs = []
cIDs = []
url = "https://" + domain + ""


print(url)

df = pd.read_csv('campids.csv')
for index, row in df.iterrows():

    payload = {'api_key':'',
                'campaign_id':'0',
                'site_offer_id':row['IDs'],
                'source_affiliate_id':'0',
                'channel_id':'0',
                'account_status_id':'0',
                'media_type_id':'0',
                'start_at_row':'0',
                'row_limit':'0',
                'sort_field':'campaign_id',
                'sort_descending':'TRUE'
            }
    print('Campaign Payload', payload)
    r = requests.get(url, params=payload)
    print(r.status_code)
    soup = BeautifulSoup(r.text, 'lxml')
    success = soup.find('success').string
    for affIDs in soup.select('campaign'):
        affID = affIDs.find('source_affiliate_id').string
        aIDs.append(affID)
        dataDict = dict()
        dataDict['offers'] = []
        affDict = {'affliate_id':aIDs}
        dataDict['offers'].append(dict(affDict))

结果如下：

dictData = {'offers': [{'affliate_id': ['9','2','45','47','14','8','30','30','2','2','9','2']}]}

我想做的是：

dictData = {'offers':[{'affiliate_id'['9','2','45','47','14','8','30','30','2','2']},{'affiliate_id':['9','2']}]}

在第一次请求时，我获得以下信息：

IDs['9','2','45','47','14','8','30','30','2','2']

在第二次请求时返回这些 ID：

['9','2']

我是 Python 新手，所以请多多包涵，就礼仪而言，我遗漏了一些东西。我很乐意提供任何其他信息。

Answer 1

这与您的初始化和附加顺序有关，这会导致您无法获得想要的结果。您将在每次迭代后覆盖您的dataDict，并插入未被覆盖的附加列表，从而为您留下一个附加了所有aIDs 的最终列表。您要做的是在 for 循环之外初始化 dataDict，然后您可以将嵌套循环中的字典附加到该列表中：

注意：在没有实际数据的情况下很难进行计算/测试，但我相信如果我在脑海中正确地计算出逻辑，就应该这样做：

import requests
import pandas as pd
from pandas import DataFrame
from bs4 import BeautifulSoup
import datetime as datetime
import json
import time


trackingDomain = ''
domain = ''

cIDs = []
url = "https://" + domain + ""

# Initialize your dictionary
dataDict = dict()

# Initialize your list in your dictionary under key `offers`
dataDict['offers'] = []

print(url)

df = pd.read_csv('campids.csv')
for index, row in df.iterrows():

    payload = {'api_key':'',
                'campaign_id':'0',
                'site_offer_id':row['IDs'],
                'source_affiliate_id':'0',
                'channel_id':'0',
                'account_status_id':'0',
                'media_type_id':'0',
                'start_at_row':'0',
                'row_limit':'0',
                'sort_field':'campaign_id',
                'sort_descending':'TRUE'
            }
    print('Campaign Payload', payload)
    r = requests.get(url, params=payload)
    print(r.status_code)
    soup = BeautifulSoup(r.text, 'lxml')
    success = soup.find('success').string

    # Initialize your list for this iteration/row in your df.iterrows
    aIDs = []
    for affIDs in soup.select('campaign'):
        affID = affIDs.find('source_affiliate_id').string

        # Append those affIDs to the aIDs list
        aIDs.append(affID)

    # Create your dictionary of key:value with key 'affiliate_id' and value the aIDs list
    affDict = {'affliate_id':aIDs}

    # NOW append that into your list in your dictionary under key `offers`
    dataDict['offers'].append(dict(affDict))