【问题标题】:R for webscraping - pull price and nameR for webscraping - 拉取价格和名称
【发布时间】:2014-10-21 13:22:18
【问题描述】:

我正在尝试从以下 URL 中的 Steam 网站获取价格和游戏名称列表,但无法弄清楚 xpathSApply 应该如何解析以下内容:

http://store.steampowered.com/search/?sort_by=Price&sort_order=ASC&';">价格

这是我的代码

require(RCurl)
require(XML)
url <- "http://store.steampowered.com/search/results?sort_by=Name&sort_order=ASC&category1=1"
SOURCE <-  getURL(url,encoding="UTF-8") #Download the page
substring (SOURCE,1,200)
PARSED <- htmlParse(SOURCE) #Format the html code 
##My problem is in this line below 
(xpathSApply(PARSED, "//div[@class='col search_price']"))

【问题讨论】:

    标签: xml r xpath xml-parsing html-parsing


    【解决方案1】:

    试试这个:

    require(RCurl)
    require(XML)
    url <- "http://store.steampowered.com/search/?sort_by=Metascore&sort_order=DESC&"
    SOURCE <-  getURL(url, encoding="UTF-8") #Download the page
    PARSED <- htmlParse(SOURCE, asText = TRUE, encoding = "utf-8")
    xpaths <- c(price="//a/div[@class='col search_price']", 
                title="//div[@class='col search_name ellipsis']/h4")
    res <- sapply(xpaths, function(x) xpathSApply(PARSED, x, xmlValue, trim = TRUE) )
    head(res)
    #      price    title                        
    # [1,] "9,99€"  "Half-Life 2"                
    # [2,] "9,99€"  "Half-Life"                  
    # [3,] "19,99€" "BioShock™"                  
    # [4,] "18,99€" "The Orange Box"             
    # [5,] "19,99€" "Portal 2"                   
    # [6,] "14,99€" "The Elder Scrolls V: Skyrim"
    

    【讨论】:

    • 不错的答案,尤其是创建标题的技术。我使用这种方法:PARSED
    • 谢谢@lawyeR。 Afaik,htmlParse 只是htmlTreeParse(useInternalNodes = TRUE, ...) 的快捷方式。我从 OP 离开了RCurl,因为它可以在需要时为您提供更好的抓取控制。
    猜你喜欢
    • 2011-12-07
    • 2023-03-26
    • 2021-05-26
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多