【问题标题】:json.loads Error: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start bytejson.loads 错误:“utf-8”编解码器无法解码位置 0 的字节 0x83:无效的起始字节
【发布时间】:2021-07-10 21:55:16
【问题描述】:

突然间,我开始收到来自此 URL 的 JSON 响应的解码错误:

https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte
Unicode error hint
The string that could not be encoded/decoded was: �� ��

我找不到任何不符合 UTF-8 的字符。

Python 3.8:

import json
import requests

url = "https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product"

headers = {
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,'
              'application/signed-exchange;v=b3;q=0.9',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
    'Connection': 'keep-alive',
    'Host': 'stockx.com',
    'TE': 'Trailers',
    'Upgrade-Insecure-Requests': '1',
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
                  'Chrome/89.0.4389.128 Safari/537.36'
}

my_response = requests.get(url, headers=headers)

if my_response.ok:
    j_data = json.loads(my_response.content)

    print(j_data)
            

注意,my_response.encoding 返回 'utf-8'

【问题讨论】:

  • 我可以确认我可以解析来自该 URL 的响应(不同的语言,不同的操作系统,但 JSON 似乎不是问题)
  • 我也尝试过并为我工作! @Everett,更像是当我直接访问编码时是“内容编码br”。所以唯一的区别就是环境!

标签: python json rest python-requests


【解决方案1】:

从我的测试来看,“Accept-Encoding”标头似乎将其强制转换为不可读的字节。如果您希望以纯文本形式发出请求,只需删除此标头并打印 my_response.text,否则,只需学习如何处理您请求的编码

【讨论】:

    【解决方案2】:

    从标题中删除Accept-Encoding。此外,您可以在响应对象上使用.json() 方法:

    import json
    import requests
    
    
    url = "https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product"
    
    headers = {
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,"
        "application/signed-exchange;v=b3;q=0.9",
        # "Accept-Encoding": "gzip, deflate, br",   # <--- remove the Accept-Encoding header
        "Accept-Language": "en-US,en;q=0.9",
        "Connection": "keep-alive",
        "Host": "stockx.com",
        "TE": "Trailers",
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/89.0.4389.128 Safari/537.36",
    }
    
    my_response = requests.get(url, headers=headers)
    if my_response.ok:
        j_data = my_response.json()
        # j_data = json.loads(my_response.content)
        print(json.dumps(j_data, indent=4))
    

    打印:

    {
        "Pagination": {
            "limit": 40,
            "page": 1,
            "total": 1,
            "lastPage": "/api/v3/browse?_search=CQ4227-030&dataType=product&order=DESC&productCategory=sneakers&sort=featured&page=1",
            "sort": [
                "featured"
            ],
            "order": [
                "DESC"
            ],
            "currentPage": "/api/v3/browse?_search=CQ4227-030&dataType=product&order=DESC&productCategory=sneakers&sort=featured",
            "nextPage": null,
            "previousPage": null
        },
        "Facets": {
            "year": {
                "2021": 1
            },
            "brand": {
                "Jordan": 1
            },
            "gender": {
                "men": 1
            },
            "dataType": {
                "product": 1
            },
            "releaseTime": {
                "1610064000": 1
            },
            "market.featured": {
                "0": 1
            },
            "productCategory": {
                "sneakers": 1
            },
            "market.lowestAsk": {
                "144": 1
            },
            "market.highestBid": {
                "170": 1
            },
            "market.volatility": {
                "0.190516": 1
            },
            "market.pricePremium": {
                "-0.206": 1
            },
            "market.deadstockSold": {
                "359": 1
            },
            "market.changePercentage": {
                "-0.046667": 1
            },
            "market.salesLast72Hours": {
                "5": 1
            },
            "market.lastLowestAskTime": {
                "1618144284": 1
            },
            "market.lastHighestBidTime": {
                "1618029125": 1
            },
            "market.averageDeadstockPrice": {
                "158": 1
            }
        },
        "Products": [
            {
                "id": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
                "uuid": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
                "brand": "Jordan",
                "breadcrumbs": [],
                "category": "Air Jordan 35",
                "charityCondition": 0,
                "childId": null,
                "colorway": "Black/Fire Red-Reflective Silver",
                "condition": "New",
                "countryOfManufacture": "CN",
                "dataType": "product",
                "description": "",
                "hidden": false,
                "listingType": "standard",
                "minimumBid": 25,
                "gender": "men",
                "doppelgangers": [],
                "media": {
                    "imageUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=700&h=500&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
                    "smallImageUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=300&h=214&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
                    "thumbUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=140&h=100&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
                    "gallery": [],
                    "hidden": false
                },
                "name": "Bred",
                "productCategory": "sneakers",
                "releaseDate": "2021-01-08",
                "releaseTime": 1610064000,
                "belowRetail": true,
                "retailPrice": 180,
                "shoe": "Jordan XXXV",
                "shortDescription": "Air-Jordan-XXXV-Bred",
                "styleId": "CQ4227-030/CQ4228-030",
                "tickerSymbol": "JB-JOXBBF",
                "title": "Jordan XXXV Bred",
                "traits": [
                    {
                        "name": "Style",
                        "value": "CQ4227-030/CQ4228-030",
                        "filterable": true,
                        "visible": true,
                        "highlight": true
                    },
                    {
                        "name": "Colorway",
                        "value": "Black/Fire Red-Reflective Silver",
                        "filterable": true,
                        "visible": true,
                        "highlight": true
                    },
                    {
                        "name": "Retail Price",
                        "value": 180,
                        "filterable": true,
                        "visible": true,
                        "highlight": false
                    },
                    {
                        "name": "Release Date",
                        "value": "2021-01-08",
                        "filterable": true,
                        "visible": true,
                        "highlight": false
                    }
                ],
                "type": 0,
                "urlKey": "air-jordan-xxxv-bred",
                "year": 2021,
                "shoeSize": null,
                "market": {
                    "productId": 0,
                    "skuUuid": "",
                    "productUuid": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
                    "lowestAsk": 144,
                    "lowestAskSize": "10.5",
                    "parentLowestAsk": 0,
                    "numberOfAsks": 148,
                    "hasAsks": 1,
                    "salesThisPeriod": 5,
                    "salesLastPeriod": 0,
                    "highestBid": 170,
                    "highestBidSize": "15",
                    "numberOfBids": 96,
                    "hasBids": 1,
                    "annualHigh": 305,
                    "annualLow": 88,
                    "deadstockRangeLow": 116,
                    "deadstockRangeHigh": 170,
                    "volatility": 0.190516,
                    "deadstockSold": 359,
                    "pricePremium": -0.206,
                    "averageDeadstockPrice": 158,
                    "lastSale": 143,
                    "lastSaleSize": "8.5",
                    "salesLast72Hours": 5,
                    "changeValue": -7,
                    "changePercentage": -0.046667,
                    "absChangePercentage": 0.046667,
                    "totalDollars": 56819,
                    "lastLowestAskTime": 1618144284,
                    "lastHighestBidTime": 1618029125,
                    "lastSaleDate": "2021-04-15T13:38:23+00:00",
                    "createdAt": "2021-01-06T19:31:31+00:00",
                    "updatedAt": 1618495984,
                    "deadstockSoldRank": 29,
                    "pricePremiumRank": 96,
                    "averageDeadstockPriceRank": 86,
                    "featured": 0
                },
                "_tags": [
                    "sneakers",
                    "jordan",
                    "air jordan",
                    "35",
                    "release_date|2021-01-08",
                    "colorway|black/fire red-reflective silver",
                    "retail_price|180",
                    "style_id|cq4227-030/cq4228-030"
                ],
                "lock_selling": false,
                "selling_countries": [
                    "AD",
                    "AE",
                    "AG",
                    "AL",
                    "AR",
                    "AS",
                    "AT",
                    "AU",
                    "BA",
                    "BB",
                    "BE",
                    "BG",
                    "BH",
                    "BM",
                    "BR",
                    "BS",
                    "BW",
                    "BZ",
                    "CH",
                    "CL",
                    "CN",
                    "CO",
                    "CR",
                    "CY",
                    "CZ",
                    "DE",
                    "DK",
                    "DM",
                    "DO",
                    "DZ",
                    "EC",
                    "EE",
                    "FI",
                    "FJ",
                    "FO",
                    "GD",
                    "GE",
                    "GF",
                    "GI",
                    "GL",
                    "GP",
                    "GR",
                    "GT",
                    "GU",
                    "HK",
                    "HN",
                    "HR",
                    "HU",
                    "ID",
                    "IE",
                    "IL",
                    "IS",
                    "JM",
                    "JO",
                    "JP",
                    "KE",
                    "KN",
                    "KR",
                    "KW",
                    "KY",
                    "KZ",
                    "LC",
                    "LI",
                    "LS",
                    "LT",
                    "LU",
                    "LV",
                    "MA",
                    "MC",
                    "MO",
                    "MP",
                    "MQ",
                    "MT",
                    "MU",
                    "MW",
                    "MX",
                    "MY",
                    "MZ",
                    "NC",
                    "NI",
                    "NL",
                    "NO",
                    "NZ",
                    "OM",
                    "PA",
                    "PE",
                    "PF",
                    "PH",
                    "PL",
                    "PR",
                    "PW",
                    "QA",
                    "RE",
                    "RO",
                    "RS",
                    "SA",
                    "SC",
                    "SE",
                    "SG",
                    "SI",
                    "SK",
                    "SN",
                    "SV",
                    "TC",
                    "TH",
                    "TW",
                    "UY",
                    "VA",
                    "VE",
                    "VI",
                    "VN",
                    "ZA",
                    "IT",
                    "PT",
                    "CA",
                    "FR",
                    "GB",
                    "GG",
                    "JE",
                    "ES",
                    "US",
                    "IN"
                ],
                "buying_countries": [
                    "AD",
                    "AE",
                    "AG",
                    "AI",
                    "AL",
                    "AM",
                    "AO",
                    "AR",
                    "AS",
                    "AT",
                    "AU",
                    "AW",
                    "BA",
                    "BB",
                    "BE",
                    "BF",
                    "BG",
                    "BH",
                    "BI",
                    "BJ",
                    "BM",
                    "BO",
                    "BR",
                    "BS",
                    "BT",
                    "BW",
                    "BY",
                    "BZ",
                    "CH",
                    "CI",
                    "CK",
                    "CL",
                    "CM",
                    "CN",
                    "CO",
                    "CR",
                    "CV",
                    "CY",
                    "CZ",
                    "DE",
                    "DJ",
                    "DK",
                    "DM",
                    "DO",
                    "DZ",
                    "EC",
                    "EE",
                    "ER",
                    "ET",
                    "FI",
                    "FJ",
                    "FO",
                    "GD",
                    "GE",
                    "GF",
                    "GI",
                    "GL",
                    "GM",
                    "GN",
                    "GP",
                    "GR",
                    "GT",
                    "GU",
                    "GW",
                    "GY",
                    "HK",
                    "HN",
                    "HR",
                    "HU",
                    "ID",
                    "IE",
                    "IL",
                    "IS",
                    "JM",
                    "JO",
                    "JP",
                    "KE",
                    "KG",
                    "KH",
                    "KI",
                    "KM",
                    "KN",
                    "KR",
                    "KW",
                    "KY",
                    "KZ",
                    "LA",
                    "LC",
                    "LI",
                    "LK",
                    "LS",
                    "LT",
                    "LU",
                    "LV",
                    "MA",
                    "MC",
                    "ME",
                    "MG",
                    "MH",
                    "MK",
                    "ML",
                    "MN",
                    "MO",
                    "MP",
                    "MQ",
                    "MR",
                    "MS",
                    "MT",
                    "MU",
                    "MV",
                    "MW",
                    "MX",
                    "MY",
                    "MZ",
                    "NC",
                    "NE",
                    "NF",
                    "NG",
                    "NI",
                    "NL",
                    "NO",
                    "NP",
                    "NZ",
                    "OM",
                    "PA",
                    "PE",
                    "PF",
                    "PG",
                    "PH",
                    "PL",
                    "PR",
                    "PW",
                    "PY",
                    "QA",
                    "RE",
                    "RO",
                    "RS",
                    "RW",
                    "SA",
                    "SB",
                    "SC",
                    "SE",
                    "SG",
                    "SI",
                    "SK",
                    "SL",
                    "SN",
                    "SR",
                    "ST",
                    "SV",
                    "SZ",
                    "TC",
                    "TD",
                    "TG",
                    "TH",
                    "TJ",
                    "TM",
                    "TN",
                    "TO",
                    "TV",
                    "TW",
                    "TZ",
                    "UA",
                    "UG",
                    "UY",
                    "VA",
                    "VC",
                    "VE",
                    "VG",
                    "VI",
                    "VN",
                    "VU",
                    "WF",
                    "WS",
                    "YE",
                    "YT",
                    "ZA",
                    "ZM",
                    "ZW",
                    "AZ",
                    "BN",
                    "CD",
                    "CG",
                    "EG",
                    "FM",
                    "GA",
                    "IT",
                    "MD",
                    "NA",
                    "PT",
                    "RU",
                    "SM",
                    "TR",
                    "TT",
                    "CA",
                    "FR",
                    "GB",
                    "GG",
                    "JE",
                    "ES",
                    "US",
                    "XK",
                    "IN"
                ],
                "objectID": "d1e93a15-3b48-431f-a98d-6f0b0ca80135"
            }
        ]
    }
    

    【讨论】:

      猜你喜欢
      • 2020-06-09
      • 1970-01-01
      • 2017-07-09
      • 2020-09-22
      • 2021-12-01
      • 2016-05-13
      • 2020-02-06
      • 1970-01-01
      相关资源
      最近更新 更多