使用R从html页面中提取数据

Question

我尝试从以下站点提取数据：

https://www.zomato.com/ncr/restaurants/north-indian

使用 R 编程，我是这个领域的学习者和初学者！

我试过这些：

> library(XML)

> doc<-htmlParse("the url mentioned above")

> Warning message:
> XML content does not seem to be XML: 'https://www.zomato.com/ncr/restaurants/north-indian' 

这是一个...我还尝试了readLines()，其输出如下：-

> readLines("the URL as mentioned above") [i can't specify more than two links so typing this]

> Error in file(con, "r") : cannot open the connection

> In addition: Warning message:

> In file(con, "r") : unsupported URL scheme

我知道该页面不是错误说明中显示的 XML，但是我有什么其他方法可以从该站点捕获数据...我确实尝试过 tidy html 将其转换为 XML 或 XHTML 然后工作它起来了，但我无处可去，也许我还不知道使用 tidy html 的实际过程！ ：（ 不确定！ 提出解决此问题的建议和更正（如果有的话）？

Answer 1

rvest 包也非常方便（建立在XML 包之上，以及其他包）：

library(rvest)

pg <- html("https://www.zomato.com/ncr/restaurants/north-indian")

# extract all the restaurant names
pg %>% html_nodes("a.result-title") %>% html_text()

##  [1] "Bukhara - ITC Maurya "                "Karim's "                            
##  [3] "Gulati "                              "Dhaba By Claridges "                 
## ...
## [27] "Dum-Pukht - ITC Maurya "              "Maal Gaadi "                         
## [29] "Sahib Sindh Sultan "                  "My Bar & Restaurant "                

# extract the ratings
pg %>% html_nodes("div.rating-div") %>% html_text() %>% gsub("[[:space:]]", "", .)

##  [1] "4.3" "4.1" "4.2" "3.9" "3.8" "4.1" "4.1" "3.4" "4.1" "4.3" "4.2" "4.2" "3.9" "3.8" "3.8" "3.4" "4.0" "3.7" "4.1"
## [20] "4.0" "3.8" "3.8" "3.9" "3.8" "4.0" "4.0" "4.7" "3.8" "3.8" "3.4"

Answer 2

我会推荐来自RCurl 包的getURL 来获取文档内容。然后我们可以用htmlParse 解析它。有时htmlParse 对某些内容有问题。在这种情况下，建议使用getURL。

url <- "https://www.zomato.com/ncr/restaurants/north-indian"

library(RCurl)
library(XML)

content <- getURL(url)
doc <- htmlParse(content)

summary(doc)
# $nameCounts
# 
#      div        a       li     span    input  article       h3     meta 
#     1337      362      232      212       33       30       30       27 
#      img   script       ul     link  section        p       br     form 
#       26       21       20       17        7        6        3        3 
#     body   footer       h1     head   header     html noscript       ol 
#        1        1        1        1        1        1        1        1 
#   strong textarea    title 
#        1        1        1 
# 
# $numNodes
# [1] 2377

另外，请注意readLines 不支持https，因此错误消息不会那么令人震惊。