【问题标题】:parse json with embedded lists into semi-long dataframe将带有嵌入列表的 json 解析为半长数据帧
【发布时间】:2018-11-09 19:12:58
【问题描述】:

我有一个包含多层嵌套的 json 文件,我正在努力将它放入一个可行的数据框中。我创建了一个基于真实结构的模拟数据的玩具示例:here 是要点。

这是我想要的输出。输出可能“更长”或具有原始 json 中的其他变量,但我显示的是核心问题。

这是 json 的一部分,它显示了我想要进入半长格式的最深嵌套级别,如上图白色所示(全宽格式就可以了)。

我已经用这个对象尝试了很多东西:

myList <- jsonlite::fromJSON("example.json", flatten=TRUE)$results

从尝试对[][[]]cbind() 进行子集化,到尝试取消嵌套嵌入列表的其他尝试。没有什么完全正确的。我会从最佳方法的建议中受益匪浅。

【问题讨论】:

    标签: r json jsonlite


    【解决方案1】:

    这能让你走得更远吗? (这是一个粗糙的结构):

    library(tidyverse)
    
    x <- (jsonlite::fromJSON("/Users/hrbrmstr/r7/gh/labs-research/2018-11-portland-ciso-event/example.json"))
    
    jsonlite::stream_out(x$results, con = gzfile("ex-res.json.gz"))
    
    y <- ndjson::stream_in("ex-res.json.gz", "tbl")
    
    gather(y, path, path_val, starts_with("path")) %>%
      gather(flow, flow_val, starts_with("flow")) %>%
      gather(name, name_val, starts_with("values.pdep")) %>%
      gather(intervention, interv_val, starts_with("values.inter")) %>%
      glimpse()
    ## Observations: 87,696
    ## Variables: 18
    ## $ contact.name <chr> "Person 1", "Person 2", "Person 1", "Person 2", "Person 1", "Person 2", "Person 1", "Person 2"...
    ## $ contact.uuid <chr> "k0dcjs", "rd3jfui", "k0dcjs", "rd3jfui", "k0dcjs", "rd3jfui", "k0dcjs", "rd3jfui", "k0dcjs", ...
    ## $ created_on   <chr> "2016-02-08T07:00:15.093813Z", "2016-02-08T07:00:15.093813Z", "2016-02-08T07:00:15.093813Z", "...
    ## $ id           <dbl> 1234, 1235, 1234, 1235, 1234, 1235, 1234, 1235, 1234, 1235, 1234, 1235, 1234, 1235, 1234, 1235...
    ## $ modified_on  <chr> "2016-02-09T04:42:54.812323Z", "2016-02-08T08:09:51.545160Z", "2016-02-09T04:42:54.812323Z", "...
    ## $ responded    <lgl> TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE...
    ## $ start.uuid   <chr> "dnxh4g", "kfj4dsi", "dnxh4g", "kfj4dsi", "dnxh4g", "kfj4dsi", "dnxh4g", "kfj4dsi", "dnxh4g", ...
    ## $ uuid         <chr> "esn4dk", "qask9dj", "esn4dk", "qask9dj", "esn4dk", "qask9dj", "esn4dk", "qask9dj", "esn4dk", ...
    ## $ exit_type    <chr> NA, "completed", NA, "completed", NA, "completed", NA, "completed", NA, "completed", NA, "comp...
    ## $ exited_on    <chr> NA, "2016-02-08T08:09:51.544998Z", NA, "2016-02-08T08:09:51.544998Z", NA, "2016-02-08T08:09:51...
    ## $ path         <chr> "path.0.node", "path.0.node", "path.0.time", "path.0.time", "path.1.node", "path.1.node", "pat...
    ## $ path_val     <chr> "ecb4cb11-6cca-4791-a950-c448e9300846", "ecb4cb11-6cca-4791-a950-c448e9300846", "2016-02-08T07...
    ## $ flow         <chr> "flow.name", "flow.name", "flow.name", "flow.name", "flow.name", "flow.name", "flow.name", "fl...
    ## $ flow_val     <chr> "weeklyratings", "weeklyratings", "weeklyratings", "weeklyratings", "weeklyratings", "weeklyra...
    ## $ name         <chr> "values.pdeps1.category", "values.pdeps1.category", "values.pdeps1.category", "values.pdeps1.c...
    ## $ name_val     <chr> "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 - 7", "0 -...
    ## $ intervention <chr> "values.intervention", "values.intervention", "values.intervention", "values.intervention", "v...
    ## $ interv_val   <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA...
    

    完整方法:

    gather(y, path, path_val, starts_with("path")) %>%
      gather(flow, flow_val, starts_with("flow")) %>%
      gather(name, name_val, starts_with("values.pdep")) %>%
      gather(intervention, interv_val, starts_with("values.inter")) %>%
      filter(grepl(".value", name)) %>% 
      filter(grepl("node", path)) %>%
      mutate(variable = gsub("values.", "", name)) %>% 
      mutate(variable = gsub(".value", "", variable)) %>% 
      distinct(contact.name, uuid, name, .keep_all = TRUE) %>% 
      select(id, uuid, contact.uuid, variable, name_val, created_on, modified_on) %>% 
      arrange(id, created_on) # optional wide %>% spread(variable, name_val)
    

    【讨论】:

    • 太棒了,@hrbmstr!我能够接受这个并在剩下的路上得到它。我应该对您的答案进行编辑以建议该方法吗?
    • 混乱作为评论,但将glimpse()替换为:filter(grepl(".value", name)) %&gt;% filter(grepl("node", path)) %&gt;% mutate(variable = gsub("values.", "", name)) %&gt;% mutate(variable = gsub(".value", "", variable)) %&gt;% distinct(contact.name, uuid, name, .keep_all = TRUE) %&gt;% select(id, uuid, contact.uuid, variable, name_val, created_on, modified_on) %&gt;% arrange(id, created_on) # optional wide %&gt;% spread(variable, name_val)
    • 一定要继续编辑它。很高兴它有帮助。过去我也有过一些粗糙的 json。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2017-03-01
    • 1970-01-01
    • 1970-01-01
    • 2020-03-24
    • 2021-01-27
    • 2015-04-11
    相关资源
    最近更新 更多