【问题标题】:Unable to scrape the names from json response无法从 json 响应中抓取名称
【发布时间】:2017-07-13 21:36:35
【问题描述】:

网页中有大约 966 个名称包含 json 内容,但使用我的脚本,我只得到了其中的 10 个。我对 json 很陌生,这就是为什么我无法弄清楚我所犯的错误。我怎样才能得到所有的名字?我正在尝试使用以下代码:

import requests

url = 'https://www.zebra.com/bin/zebra/partnersearch?inMiles=true&start=0&numRows=10&latitude=39.5500507&longitude=-105.7820674&sortOrder=asc&sortBy=distance&country=US&searchRadius=5000'

response = requests.get(url)
data = response.json()
for item in data:
    print(item['name'])

该页面的部分 json 内容是:

[{"id":"001i000001XR9dqAAD","website":"www.resortinternet.com","type":"partner","phoneNumber":"+1.970.262.3515","name":"Resortnet, LLC","logoPresent":"No","logoExtension":"","des":"Technology provider for destination resorts","translatedName":"ResortInternet","dbaName":"ResortInternet","PR":"NA","AN":"6244306","accountType":["Reseller"],"contentType":"parent","countries":["US"],"HSA":[],"countriesAndHsa":["US"],"premierSolutionPartner":false,"premierBusinessPartner":false,"solutionPartner":true,"businessPartner":false,"advancedSpecialistBarcodePrinterSupplies":false,"advancedSpecialistCardPrinters":false,"advancedSpecialistSupplies":false,"advancedSpecialistWirelessNetworks":false,"advancedSpecialistPrintEngines":false,"advancedSpecialistRfid":false,"specialistBarcodePrinterSupplies":false,"specialistCardPrinters":false,"specialistSupplies":false,"specialistWirelessNetworks":false,"specialistPrintEngines":false,"specialistRfid":false,"advancedRepairSpecialistLabelPrinter":false,"advancedRepairSpecialistCardPrinter":false,"advancedRepairSpecialistMobilePrinter":false,"advancedRepairSpecialistPrintEngine":false,"repairSpecialistLabelPrinter":false,"repairSpecialistCardPrinter":false,"repairSpecialistMobilePrinter":false,"repairSpecialistPrintEngine":false,"registeredResellerNoSpecialization":false,"pmiWraps":[{"programName":"Solution Partner","category":"Reseller","id":"001i000001XR9dqAAD_2","type":"pmiWrap","contentType":"child"}],"partnerLocations":[{"locationType":"Headquarters","addressLine1":"117 S 6th Ave.,","addressLine2":"PO Box 2718","city":"Frisco","state":"Colorado","zipCode":"80443","country":"United States","phone":"(970) 262-3515","fax":"(970) 668-9431","latlon":"39.5754576,-106.0952117","distance":16.8,"countryCode":"US","HSA":[],"id":"001i000001XR9dqAAD_0","type":"partnerLocation","contentType":"child"},{"locationType":"Primary Location","addressLine1":"117 S 6th Ave.,","city":"Frisco","state":"Colorado","zipCode":"80443","country":"United States","phone":"+1.970.262.3515","latlon":"39.5754576,-106.0952117","distance":16.8,"countryCode":"US","HSA":[],"id":"001i000001XR9dqAAD_1","type":"partnerLocation","contentType":"child"},{"locationType":"Address","addressLine1":"RESORTINTERNET\r2718:FRISCO:80443\r117 S 6TH AVERM UNIT 2","city":"Frisco","state":"Colorado","stateCode":"CO","zipCode":"80443","country":"United States","latlon":"39.5744309,-106.0975203","distance":16.9,"countryCode":"US","HSA":[],"id":"001i000001XR9dqAAD_100","type":"partnerLocation","contentType":"child"}],"verticalHierarchyWraps":[],"primaryLocation":{"locationType":"Headquarters","addressLine1":"117 S 6th Ave.,","addressLine2":"PO Box 2718","city":"Frisco","state":"Colorado","zipCode":"80443","country":"United States","phone":"(970) 262-3515","fax":"(970) 668-9431","latlon":"39.5754576,-106.0952117","distance":16.8,"countryCode":"US","HSA":

【问题讨论】:

    标签: json python-3.x web-scraping


    【解决方案1】:

    我认为您的代码没有问题。如果您检查len(data),它会返回10,这意味着结果列表仅包含10 个(大)JSON 对象。

    是否有某些原因您期望超过 10 个,或者您是否尝试访问这些较大对象中的每个对象的 name 属性?

    【讨论】:

    • 感谢 Emersonct,您的回答。我也检查了长度。现在,作为首发,这对我来说更有意义。在这一点上,我想我应该想到硒。
    • @SMth80:您实际上不需要这样做来获得更多结果。请注意,在您使用的 url 中有查询字符串参数:&numRows=10。将其更改为更大的数字(或删除它?)应该会有所帮助。
    • 如果这还不够,我认为您可以通过每次调用将start 查询参数增加numRows 数量来循环所有结果,直到不再返回结果。
    【解决方案2】:

    您的 JSON 是一个对象数组,因此当您遍历数据时,您不会在 item 变量中获取数组,而是获取数组的索引。

    您可以通过使用 item 变量作为索引来获取数组,一旦您引用了该数组,您就可以读取属性对象,例如名称:

    像这样:

    for index in data:
        item = data[index]
        print(item['name'])
    

    这里是 JavaScript:

    <script>
    var data = [
       {
          "id":"001i000001XR9dqAAD",
          "website":"www.resortinternet.com",
          "type":"partner",
          "phoneNumber":"+1.970.262.3515",
          "name":"Resortnet, LLC",
          "logoPresent":"No",
          "logoExtension":"",
          "des":"Technology provider for destination resorts",
          "translatedName":"ResortInternet",
          "dbaName":"ResortInternet",
          "PR":"NA",
          "AN":"6244306",
          "accountType":[
             "Reseller"
          ],
          "contentType":"parent",
          "countries":[
             "US"
          ],
          "HSA":[
    
          ],
          "countriesAndHsa":[
             "US"
          ],
          "premierSolutionPartner":false,
          "premierBusinessPartner":false,
          "solutionPartner":true,
          "businessPartner":false,
          "advancedSpecialistBarcodePrinterSupplies":false,
          "advancedSpecialistCardPrinters":false,
          "advancedSpecialistSupplies":false,
          "advancedSpecialistWirelessNetworks":false,
          "advancedSpecialistPrintEngines":false,
          "advancedSpecialistRfid":false,
          "specialistBarcodePrinterSupplies":false,
          "specialistCardPrinters":false,
          "specialistSupplies":false,
          "specialistWirelessNetworks":false,
          "specialistPrintEngines":false,
          "specialistRfid":false,
          "advancedRepairSpecialistLabelPrinter":false,
          "advancedRepairSpecialistCardPrinter":false,
          "advancedRepairSpecialistMobilePrinter":false,
          "advancedRepairSpecialistPrintEngine":false,
          "repairSpecialistLabelPrinter":false,
          "repairSpecialistCardPrinter":false,
          "repairSpecialistMobilePrinter":false,
          "repairSpecialistPrintEngine":false,
          "registeredResellerNoSpecialization":false,
          "pmiWraps":[
             {
                "programName":"Solution Partner",
                "category":"Reseller",
                "id":"001i000001XR9dqAAD_2",
                "type":"pmiWrap",
                "contentType":"child"
             }
          ],
          "partnerLocations":[
             {
                "locationType":"Headquarters",
                "addressLine1":"117 S 6th Ave.,",
                "addressLine2":"PO Box 2718",
                "city":"Frisco",
                "state":"Colorado",
                "zipCode":"80443",
                "country":"United States",
                "phone":"(970) 262-3515",
                "fax":"(970) 668-9431",
                "latlon":"39.5754576,-106.0952117",
                "distance":16.8,
                "countryCode":"US",
                "HSA":[
    
                ],
                "id":"001i000001XR9dqAAD_0",
                "type":"partnerLocation",
                "contentType":"child"
             },
             {
                "locationType":"Primary Location",
                "addressLine1":"117 S 6th Ave.,",
                "city":"Frisco",
                "state":"Colorado",
                "zipCode":"80443",
                "country":"United States",
                "phone":"+1.970.262.3515",
                "latlon":"39.5754576,-106.0952117",
                "distance":16.8,
                "countryCode":"US",
                "HSA":[
    
                ],
                "id":"001i000001XR9dqAAD_1",
                "type":"partnerLocation",
                "contentType":"child"
             },
             {
                "locationType":"Address",
                "addressLine1":"RESORTINTERNET\r2718:FRISCO:80443\r117 S 6TH AVERM UNIT 2",
                "city":"Frisco",
                "state":"Colorado",
                "stateCode":"CO",
                "zipCode":"80443",
                "country":"United States",
                "latlon":"39.5744309,-106.0975203",
                "distance":16.9,
                "countryCode":"US",
                "HSA":[
    
                ],
                "id":"001i000001XR9dqAAD_100",
                "type":"partnerLocation",
                "contentType":"child"
             }
          ],
          "verticalHierarchyWraps":[
    
          ],
          "primaryLocation":{
             "locationType":"Headquarters",
             "addressLine1":"117 S 6th Ave.,",
             "addressLine2":"PO Box 2718",
             "city":"Frisco",
             "state":"Colorado",
             "zipCode":"80443",
             "country":"United States",
             "phone":"(970) 262-3515",
             "fax":"(970) 668-9431",
             "latlon":"39.5754576,-106.0952117",
             "distance":16.8,
             "countryCode":"US"
          }
       }
    ];
     for (var index in data)
     {
       
       var item=data[index];
       console.log(item["name"]);
       console.log(item);
    
     }
     </script>

    【讨论】:

    • 感谢您的回复亚历山大希金斯。它没有成功。我现在收到此错误:“列表索引必须是整数或切片,而不是字典”
    猜你喜欢
    • 2018-06-26
    • 1970-01-01
    • 2018-01-28
    • 2021-04-08
    • 1970-01-01
    • 2013-08-23
    • 1970-01-01
    • 1970-01-01
    • 2021-11-26
    相关资源
    最近更新 更多