lxml web-scraping返回空值

Question

我正在尝试从该站点获取所有食物类别https://www.walmart.com/cp/976759 这是类别容器的快照

<div id="cp-center-module-5" class="cp-center-module"><span style="font-size: 0px;"></span><div data-module="FeaturedCategoriesCollapsible" data-module-id="e05783ed-f2bb-44f3-956f-9d7d5286d25b" class="TempoTileCollapsible FeaturedCategoriesCollapsible" data-tl-id="categorypage-FeaturedCategoriesCollapsible"><div class="TempoTileCollapsible-header"><div class="ModuleHeader text-center" data-tl-id="categorypage-FeaturedCategoriesCollapsible-ModuleHeader"><h2 class="ModuleHeader-heading font-normal display-inline-block" data-tl-id="categorypage-FeaturedCategoriesCollapsible-ModuleHeader-title">Shop by Category</h2></div><div class="TempoTileCollapsible-expanderToggle display-block"><button class="button TempoTileCollapsible-button display-inline-block font-normal button--link" data-automation-id="button" data-tl-id="categorypage-FeaturedCategoriesCollapsible-expander" aria-controls="e05783ed-f2bb-44f3-956f-9d7d5286d25b-expander" aria-expanded="true" aria-label="See Less categories" type="button"><span class="button-wrapper"><span class="TempoTileCollapsible-expandTitle">See Less</span><span aria-hidden="true" class="elc-icon xs-padding-sides valign-top elc-icon-minus"></span></span></button></div></div><div class="TempoTileCollapsible-expander expanded" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-expander" aria-expanded="true"><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-0"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-4fec/k2-_d0c27367-0903-424d-9ed7-25ff31ed2078.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Coffee" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Coffee</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Coffee" data-uid="EVz8WxyK" href="/cp/coffee/1086446?povid=976759+%7C+2018-12-26+%7C+Food%20Coffee%20Shop%20by%20Category%20Tile%201" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-0" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-0-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-1" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-1"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-b006/k2-_9c1d502f-c08d-4591-a734-b205d0ffe45b.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Meal Solutions, Grains & Pasta" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Meal Solutions, Grains & Pasta</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Meal Solutions, Grains & Pasta" data-uid="a0xEKGc1" href="/cp/meal-solutions-grains-pasta/976794?povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Shop%20by%20Category%20Tile%202" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-1" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-1-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-2" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-2"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-66f5/k2-_a622db4c-a789-4f03-bf16-440ad12efcd8.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Snacks" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Snacks</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Snacks" data-uid="iYpawUR8" href="/cp/snacks-cookies-chips/976787?povid=976759+%7C+2018-12-26+%7C+Food%20Snack%20Shop%20by%20Category%20Tile%203" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-2" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-2-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-3" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-3"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-b691/k2-_95cdb69e-5175-408a-b18e-7c8a4902da65.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Beverages" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Beverages</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Beverages" data-uid="YDQP7Zs1" href="/cp/beverages/976782?povid=976759+%7C+2018-12-26+%7C+Food%20Beverages%20Shop%20by%20Category%20Tile%204" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-3" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-3-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-4" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-4"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-26df/k2-_0e1ed3ed-51c5-4d55-a4b3-64d5beab75c4.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Chocolate, Candy & Gum" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Chocolate, Candy & Gum</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Chocolate, Candy & Gum" data-uid="khYex7Z3" href="/cp/chocolate-candy-gum/1096070?povid=976759+%7C+2018-12-26+%7C+Food%20Candy%20Shop%20by%20Category%20Tile%205" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-4" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-4-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-5" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-5"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-c487/k2-_0b0b1864-112c-4323-9474-9556739bf3b5.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Condiments" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Condiments</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Condiments" data-uid="rxFVAq08" href="/cp/976786?povid=976759+%7C+2018-12-26+%7C+Food%20Condiments%20Shop%20by%20Category%20Tile%206" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-5" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-5-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-6" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-6"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-26b5/k2-_7ad38e98-0ccd-479f-9bfa-1d4d4dfe90a2.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Baking" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Baking</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Baking" data-uid="pI4YqGyq" href="/cp/baking/976780?povid=976759+%7C+2018-12-26+%7C+Food%20Baking%20Shop%20by%20Category%20Tile%207" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-6" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-6-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-7" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-7"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-f53c/k2-_3a8d9006-e514-48b7-ad81-c4a70a8d39e9.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Breakfast & Cereal" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Breakfast & Cereal</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Breakfast & Cereal" data-uid="PQUXkqiQ" href="/cp/breakfast-food-cereal/976783?povid=976759+%7C+2018-12-26+%7C+Food%20Breakfast%20&%20Cereal%20Shop%20by%20Category%20Tile%208" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-7" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-7-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-8" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-8"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-8e1b/k2-_3e651309-806d-4633-95f6-ec015c783759.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Food Gift Baskets" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Food Gift Baskets</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Food Gift Baskets" data-uid="nEwGTdfg" href="/browse/food/gift-baskets/976759_1089004?povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209&povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-8" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-8-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-9" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-9"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-369d/k2-_5fc9cbf1-4b2e-47ba-a35c-d5016d80a0a1.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Emergency Food" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Emergency Food</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Emergency Food" data-uid="L0VEhGaa" href="/browse/meal-solutions-grains-pasta/emergency-food/976759_976794_1094144?povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-9" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-9-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-10" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-10"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-e889/k2-_025af29b-a175-43d9-a7f1-8a41b7f595d8.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Organic Foods" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Organic Foods</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Organic Foods" data-uid="VodpOeXr" href="/browse/food/organic-foods/976759_1228024?povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-10" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-10-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-11" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-11"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-40b6/k2-_d4adeded-bc5a-4141-8ff5-484e5a57af7b.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Gluten-Free Foods" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Gluten-Free Foods</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Gluten-Free Foods" data-uid="-BPKl3mO" href="/browse/food/gluten-free-foods/976759_1228023?povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-11" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-11-link" style="background-image: url("about:blank");"></a></div><div role="link" tabindex="0" class="TempoCategoryTile Grid-col u-size-1-2 u-size-1-4-m u-size-1-4-xl display-inline-block" id="TempoCategoryTile-e05783ed-f2bb-44f3-956f-9d7d5286d25b-tile-12" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-12"><div class="TempoCategoryTile-tile valign-top"><div class="TempoCategoryTile-imgContainer" aria-hidden="true"><div style="padding-bottom: 100%; height: 0px; width: 100%; position: relative;"><img aria-hidden="true" tabindex="-1" itemprop="image" loading="lazy" src="//i5.walmartimages.com/dfw/4ff9c6c9-bed9/k2-_b82a8177-43e5-45d2-bc92-1dccd94d1e5d.v1.jpg?odnWidth=282&odnHeight=282&odnBg=ffffff" alt="Meal Delivery Services" class="TempoCategoryTile-img" style="position: absolute; width: 100%; left: 0px; top: 0px;"></div></div><div class="TempoCategoryTile-content valign-top" aria-hidden="true"><span class="TempoCategoryTile-linkText u-text-truncate">Meal Delivery Services</span><div class=""></div></div></div><a class="TempoCategoryTile-overlayLink u-focusTile" aria-hidden="false" aria-label="Meal Delivery Services" data-uid="XAyoF2GU" href="/browse/food/meal-kits-specialty-food-boxes/976759_7123943?povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010" id="e05783ed-f2bb-44f3-956f-9d7d5286d25b-tileLink-12" tabindex="0" data-tl-id="categorypage-FeaturedCategoriesCollapsible-TempoCategoryTile-12-link" style="background-image: url("about:blank");"></a></div><a class="TempoTileCollapsible-seeAllCategoriesButton font-bold" aria-label="Shop All" data-uid="t1WJY_mN" href="https://www.walmart.com/browse/food/976759" data-tl-id="categorypage-FeaturedCategoriesCollapsible-SeeAllCategoriesButton">Shop All</a></div></div></div>

这是我的代码

import requests
import lxml.html
from urllib.request import urlopen

Walmartfoods = 'https://www.walmart.com/cp/976759'

def check():
    data = urlopen(Walmartfoods).read(); #TopGear as a test
    return data.decode('utf-8', 'ignore');


doc = lxml.html.document_fromstring(check())
 

#This will create a list of buyers:
foodcategories = doc.xpath("//div[contains(@class, 'TempoCategoryTile-tile')]")
print(foodcategories)

我想获取类别名称、链接和图像，但第一步是返回空。我做错了什么？

Answer 1

尽管 OP 要求使用 lxml 进行解析，但此答案使用 beautifulsoup。

当您看到网站时，所有数据都是从脚本标签加载的。因此，完整的数据存储在一个脚本标签中，id 为category。

import requests, json
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36'}

res = requests.get("https://www.walmart.com/cp/976759", headers=headers)
soup = BeautifulSoup(res.text, "html.parser")

script = soup.find("script", {"id":"category"})

data = json.loads(script.get_text(strip=True))

with open("data.json", "w") as f:
    json.dump(data, f)

上述脚本会将所有数据保存到一个 json 文件中。这是一个很大的json。在那里您可以获取类别的链接和图像。

大 json 中类别的输出。

[
    {
        "image": {
            "alt": "Coffee",
            "assetId": "24073832",
            "assetName": "42227-230294--Food_Coffee_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/coffee/1086446?povid=976759+%7C+2018-12-26+%7C+Food%20Coffee%20Shop%20by%20Category%20Tile%201",
                "rawValue": "/cp/coffee/1086446",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Coffee%20Shop%20by%20Category%20Tile%201"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-4fec/k2-_d0c27367-0903-424d-9ed7-25ff31ed2078.v1.jpg",
            "title": "Coffee",
            "width": "320",
            "size": "48986",
            "contentType": "image/jpg",
            "uid": "EVz8WxyK"
        },
        "uid": "KzzTghKO"
    },
    {
        "image": {
            "alt": "Meal Solutions, Grains & Pasta",
            "assetId": "16511345",
            "assetName": "41423-209368-Food-Meals_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/meal-solutions-grains-pasta/976794?povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Shop%20by%20Category%20Tile%202",
                "rawValue": "/cp/meal-solutions-grains-pasta/976794",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Shop%20by%20Category%20Tile%202"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-b006/k2-_9c1d502f-c08d-4591-a734-b205d0ffe45b.v1.jpg",
            "title": "Meal Solutions, Grains & Pasta",
            "width": "320",
            "size": "21747",
            "contentType": "image/jpg",
            "uid": "a0xEKGc1"
        },
        "uid": "Sa4hkgg8"
    },
    {
        "image": {
            "alt": "Snacks",
            "assetId": "16511346",
            "assetName": "41423-209369-Food_Snacks_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/snacks-cookies-chips/976787?povid=976759+%7C+2018-12-26+%7C+Food%20Snack%20Shop%20by%20Category%20Tile%203",
                "rawValue": "/cp/snacks-cookies-chips/976787",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Snack%20Shop%20by%20Category%20Tile%203"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-66f5/k2-_a622db4c-a789-4f03-bf16-440ad12efcd8.v1.jpg",
            "title": "Snacks",
            "width": "320",
            "size": "22038",
            "contentType": "image/jpg",
            "uid": "iYpawUR8"
        },
        "uid": "KN0Y6XJk"
    },
    {
        "image": {
            "alt": "Beverages",
            "assetId": "31886230",
            "assetName": "42592-239546 Food Beverages Featured Category Tile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/beverages/976782?povid=976759+%7C+2018-12-26+%7C+Food%20Beverages%20Shop%20by%20Category%20Tile%204",
                "rawValue": "/cp/beverages/976782",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Beverages%20Shop%20by%20Category%20Tile%204"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-b691/k2-_95cdb69e-5175-408a-b18e-7c8a4902da65.v1.jpg",
            "title": "Beverages",
            "width": "320",
            "size": "21411",
            "contentType": "image/jpg",
            "uid": "YDQP7Zs1"
        },
        "uid": "eronFjMz"
    },
    {
        "image": {
            "alt": "Chocolate, Candy & Gum",
            "assetId": "16511348",
            "assetName": "41423-209371-Food_Candy_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/chocolate-candy-gum/1096070?povid=976759+%7C+2018-12-26+%7C+Food%20Candy%20Shop%20by%20Category%20Tile%205",
                "rawValue": "/cp/chocolate-candy-gum/1096070",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Candy%20Shop%20by%20Category%20Tile%205"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-26df/k2-_0e1ed3ed-51c5-4d55-a4b3-64d5beab75c4.v1.jpg",
            "title": "Chocolate, Candy & Gum",
            "width": "320",
            "size": "24819",
            "contentType": "image/jpg",
            "uid": "khYex7Z3"
        },
        "uid": "N57hxj54"
    },
    {
        "image": {
            "alt": "Condiments",
            "assetId": "16511349",
            "assetName": "41423-209372-Food_Condiments_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/976786?povid=976759+%7C+2018-12-26+%7C+Food%20Condiments%20Shop%20by%20Category%20Tile%206",
                "rawValue": "/cp/976786",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Condiments%20Shop%20by%20Category%20Tile%206"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-c487/k2-_0b0b1864-112c-4323-9474-9556739bf3b5.v1.jpg",
            "title": "Condiments",
            "width": "320",
            "size": "12514",
            "contentType": "image/jpg",
            "uid": "rxFVAq08"
        },
        "uid": "Ych6vXbE"
    },
    {
        "image": {
            "alt": "Baking",
            "assetId": "16511350",
            "assetName": "41423-209373-Food_Baking_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/baking/976780?povid=976759+%7C+2018-12-26+%7C+Food%20Baking%20Shop%20by%20Category%20Tile%207",
                "rawValue": "/cp/baking/976780",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Baking%20Shop%20by%20Category%20Tile%207"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-26b5/k2-_7ad38e98-0ccd-479f-9bfa-1d4d4dfe90a2.v1.jpg",
            "title": "Baking",
            "width": "320",
            "size": "18935",
            "contentType": "image/jpg",
            "uid": "pI4YqGyq"
        },
        "uid": "07562lCu"
    },
    {
        "image": {
            "alt": "Breakfast & Cereal",
            "assetId": "16511351",
            "assetName": "41423-209374-Food_Breakfast_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/cp/breakfast-food-cereal/976783?povid=976759+%7C+2018-12-26+%7C+Food%20Breakfast%20&%20Cereal%20Shop%20by%20Category%20Tile%208",
                "rawValue": "/cp/breakfast-food-cereal/976783",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Breakfast%20&%20Cereal%20Shop%20by%20Category%20Tile%208"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-f53c/k2-_3a8d9006-e514-48b7-ad81-c4a70a8d39e9.v1.jpg",
            "title": "Breakfast & Cereal",
            "width": "320",
            "size": "24847",
            "contentType": "image/jpg",
            "uid": "PQUXkqiQ"
        },
        "uid": "gJmzhaYu"
    },
    {
        "image": {
            "alt": "Food Gift Baskets",
            "assetId": "16511356",
            "assetName": "41423-209379-Food_GiftBaskets_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/browse/food/gift-baskets/976759_1089004?povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209&povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209",
                "rawValue": "/browse/food/gift-baskets/976759_1089004?povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Gift%20Baskets%20Shop%20by%20Category%20Tile%209"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-8e1b/k2-_3e651309-806d-4633-95f6-ec015c783759.v1.jpg",
            "title": "Food Gift Baskets",
            "width": "320",
            "size": "19695",
            "contentType": "image/jpg",
            "uid": "nEwGTdfg"
        },
        "uid": "ommZYX3q"
    },
    {
        "image": {
            "alt": "Emergency Food",
            "assetId": "16511354",
            "assetName": "41423-209377-Food_EmergencyFood_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/browse/meal-solutions-grains-pasta/emergency-food/976759_976794_1094144?povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010",
                "rawValue": "/browse/meal-solutions-grains-pasta/emergency-food/976759_976794_1094144?povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Emergency%20Food%20Shop%20by%20Category%20Tile%2010"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-369d/k2-_5fc9cbf1-4b2e-47ba-a35c-d5016d80a0a1.v1.jpg",
            "title": "Emergency Food",
            "width": "320",
            "size": "12594",
            "contentType": "image/jpg",
            "uid": "L0VEhGaa"
        },
        "uid": "EV4aR1IJ"
    },
    {
        "image": {
            "alt": "Organic Foods",
            "assetId": "16511352",
            "assetName": "41423-209375-Food_Organic_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/browse/food/organic-foods/976759_1228024?povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010",
                "rawValue": "/browse/food/organic-foods/976759_1228024?povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Organic%20Foods%20Shop%20by%20Category%20Tile%2010"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-e889/k2-_025af29b-a175-43d9-a7f1-8a41b7f595d8.v1.jpg",
            "title": "Organic Foods",
            "width": "320",
            "size": "14996",
            "contentType": "image/jpg",
            "uid": "VodpOeXr"
        },
        "uid": "6bskKrLd"
    },
    {
        "image": {
            "alt": "Gluten-Free Foods",
            "assetId": "16511353",
            "assetName": "41423-209376-Food_Gluten-Free_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/browse/food/gluten-free-foods/976759_1228023?povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010",
                "rawValue": "/browse/food/gluten-free-foods/976759_1228023?povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Gluten%20Free%20Shop%20by%20Category%20Tile%2010"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-40b6/k2-_d4adeded-bc5a-4141-8ff5-484e5a57af7b.v1.jpg",
            "title": "Gluten-Free Foods",
            "width": "320",
            "size": "11866",
            "contentType": "image/jpg",
            "uid": "-BPKl3mO"
        },
        "uid": "DxMgKndk"
    },
    {
        "image": {
            "alt": "Meal Delivery Services",
            "assetId": "16511355",
            "assetName": "41423-209378-Food_MealKits_FeaturedCategoryTile_V1.jpg",
            "clickThrough": {
                "type": "url",
                "value": "/browse/food/meal-kits-specialty-food-boxes/976759_7123943?povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010&povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010",
                "rawValue": "/browse/food/meal-kits-specialty-food-boxes/976759_7123943?povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010",
                "tag": "povid=976759+%7C+2018-12-26+%7C+Food%20Meal%20Delivery%20Services%20Shop%20by%20Category%20Tile%2010"
            },
            "height": "320",
            "src": "https://i5.walmartimages.com/dfw/4ff9c6c9-bed9/k2-_b82a8177-43e5-45d2-bc92-1dccd94d1e5d.v1.jpg",
            "title": "Meal Delivery Services",
            "width": "320",
            "size": "20720",
            "contentType": "image/jpg",
            "uid": "XAyoF2GU"
        },
        "uid": "MeJ6LK_Z"
    }
]

更新： 为了从 json 中获取类别链接和其他信息： 假设json存储在data变量中

for innerjson in data["category"]["presoData"]["modules"]["center"]:
    if "moduleData" in innerjson and "title" in innerjson["moduleData"]["configs"] and innerjson["moduleData"]["configs"]["title"] == "Shop by Category":
        print(innerjson["moduleData"]["configs"]["categories"])