【问题标题】:r rbind a large list of vectors created unwanted extra columnsr rbind 大量向量创建了不需要的额外列
【发布时间】:2018-08-11 18:59:55
【问题描述】:

我正在尝试将大量字符向量(2284879 个元素和 593.7 Mb)转换为数据帧。每个列表元素都是一个包含四个字符串的字符向量——这些字符串是从一个 4-gram 列表创建的。

class(words_split)
[1] "list"
length(words_split)
[1] 2284879
head(words_split)
[[1]]
[1] "the" "end" "of"  "the"
[[2]]
[1] "the"  "rest" "of"   "the" 
[[3]]
[1] "at"  "the" "end" "of" 
[[4]]
[1] "to"  "be"  "abl" "to" 
[[5]]
[1] "at"   "the"  "same" "time"
[[6]]
[1] "in"    "the"   "middl" "of" 

期望的结果是:

    [,1]  [,2]   [,3]  [,4] 
[1,] "the" "end"  "of"  "the"
[2,] "the" "rest" "of"  "the"
[3,] "at"  "the"  "end" "of" 
[4,] "to"  "be"   "abl" "to" 

搜索并尝试了各种方法后,似乎do.callrbing 是解决方案。

words_table<-as.data.table(do.call(rbind,words_split))

但结果有 12 列,而不是 4 列:

    [,1]  [,2]   [,3]    [,4]   [,5]  [,6]   [,7]    [,8]   [,9]  [,10]  [,11]   [,12] 
[1,] "the" "end"  "of"    "the"  "the" "end"  "of"    "the"  "the" "end"  "of"    "the" 
[2,] "the" "rest" "of"    "the"  "the" "rest" "of"    "the"  "the" "rest" "of"    "the" 
[3,] "at"  "the"  "end"   "of"   "at"  "the"  "end"   "of"   "at"  "the"  "end"   "of"  
[4,] "to"  "be"   "abl"   "to"   "to"  "be"   "abl"   "to"   "to"  "be"   "abl"   "to"  
[5,] "at"  "the"  "same"  "time" "at"  "the"  "same"  "time" "at"  "the"  "same"  "time"
[6,] "in"  "the"  "middl" "of"   "in"  "the"  "middl" "of"   "in"  "the"  "middl" "of"  

如果我对words_split 的一部分进行采样,比如前 4 个元素,然后做同样的事情,结果很好:

> words_head<-words_split[1:4]
> words_head
[[1]]
[1] "the" "end" "of"  "the"

[[2]]
[1] "the"  "rest" "of"   "the" 

[[3]]
[1] "at"  "the" "end" "of" 

[[4]]
[1] "to"  "be"  "abl" "to" 
> class(words_head[1])
[1] "list"
> class(words_head[[1]])
[1] "character"
> words_head[[1]]
[1] "the" "end" "of"  "the"
> words_head_comb<-do.call(rbind,words_head)
print(head(words_head_comb))
     [,1]  [,2]   [,3]  [,4] 
[1,] "the" "end"  "of"  "the"
[2,] "the" "rest" "of"  "the"
[3,] "at"  "the"  "end" "of" 
[4,] "to"  "be"   "abl" "to" 

为什么rbind() 会重复合并我的列表两次,当列表很大时,当列表很小时,它似乎工作?

【问题讨论】:

  • 列表中的一个向量可能有 12 个元素而不是 4 个。rbinding 时,重复只有 4 个元素的行以达到 12 的大小。尝试table(lengths(word_split)) 有了解列表中向量的长度。
  • 返回 12 列时是否收到任何警告?你确定你的列表元素每个都有 4 个值吗?
  • 感谢@Lamia 和@AntoniosK。我刚查了一下,发现有 124 个元素的长度从 5 到 12。我不知道这是怎么发生的,因为我使用了 quenteda 包中的 dfm 函数来创建 4gram,所以我假设所有标记都是四个单词用破折号连接的。我猜有些词原本已经包括_。我使用while 删除所有这些不规则元素,然后使用rbind。这次成功了。
  • FWIW:do.call(rbind, words_split) 创建一个矩阵,而不是 OP 要求的 data.frame。
  • @Uwe,是的,感谢您指出这一点——实际上我的实际代码中确实有 as.data.table 包裹了 do.call。我主要关心的是如何将所有列表组合成一个四列可搜索数据表以进行进一步处理。我现在将对其进行编辑。

标签: r rbind do.call


【解决方案1】:

正如 cmets 中所指出的,当所有元素的长度都为 4 时,结果将是正确的。例如,

words_split <- list(
  c("the", "end", "of", "the"),
  c("the", "rest", "of", "the" ),
  c("at", "the", "end", "of"), 
  c("to", "be", "abl", "to"), 
  c("at", "the", "same", "time"), 
  c("in", "the", "middl", "of"))

do.call(rbind,words_split)
#R       [,1]  [,2]   [,3]    [,4]  
#R [1,] "the" "end"  "of"    "the" 
#R [2,] "the" "rest" "of"    "the" 
#R [3,] "at"  "the"  "end"   "of"  
#R [4,] "to"  "be"   "abl"   "to"  
#R [5,] "at"  "the"  "same"  "time"
#R [6,] "in"  "the"  "middl" "of"  

但是,如果其中一个元素长于四个,那么来自help("rbind") 的以下句子很重要

如果所有参数都是向量,则结果中的列(行)数等于最长向量的长度。较短参数中的值将被回收以达到此长度(如果仅部分地回收它们,则使用warning)。

所以如果添加一个长度为 12 的元素,那么我们得到 ​​p>

words_split[[7]] <- c(
  "some", "char", "sequence", "which", "has", "length", "of", 
  "eights", "which", "is", "too", "long")
do.call(rbind,words_split)
#R      [,1]   [,2]   [,3]       [,4]    [,5]  [,6]     [,7]    [,8]    
#R [1,] "the"  "end"  "of"       "the"   "the" "end"    "of"    "the"   
#R [2,] "the"  "rest" "of"       "the"   "the" "rest"   "of"    "the"   
#R [3,] "at"   "the"  "end"      "of"    "at"  "the"    "end"   "of"    
#R [4,] "to"   "be"   "abl"      "to"    "to"  "be"     "abl"   "to"    
#R [5,] "at"   "the"  "same"     "time"  "at"  "the"    "same"  "time"  
#R [6,] "in"   "the"  "middl"    "of"    "in"  "the"    "middl" "of"    
#R [7,] "some" "char" "sequence" "which" "has" "length" "of"    "eights"
#R      [,9]    [,10]  [,11]   [,12] 
#R [1,] "the"   "end"  "of"    "the" 
#R [2,] "the"   "rest" "of"    "the" 
#R [3,] "at"    "the"  "end"   "of"  
#R [4,] "to"    "be"   "abl"   "to"  
#R [5,] "at"    "the"  "same"  "time"
#R [6,] "in"    "the"  "middl" "of"  
#R [7,] "which" "is"   "too"   "long"

没有警告,因为 12 是 4 的倍数。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2020-04-04
    • 1970-01-01
    • 2013-10-09
    • 2018-07-08
    • 2018-12-31
    • 2018-08-23
    • 2021-04-07
    • 2019-05-07
    相关资源
    最近更新 更多