【发布时间】:2018-08-11 18:59:55
【问题描述】:
我正在尝试将大量字符向量(2284879 个元素和 593.7 Mb)转换为数据帧。每个列表元素都是一个包含四个字符串的字符向量——这些字符串是从一个 4-gram 列表创建的。
class(words_split)
[1] "list"
length(words_split)
[1] 2284879
head(words_split)
[[1]]
[1] "the" "end" "of" "the"
[[2]]
[1] "the" "rest" "of" "the"
[[3]]
[1] "at" "the" "end" "of"
[[4]]
[1] "to" "be" "abl" "to"
[[5]]
[1] "at" "the" "same" "time"
[[6]]
[1] "in" "the" "middl" "of"
期望的结果是:
[,1] [,2] [,3] [,4]
[1,] "the" "end" "of" "the"
[2,] "the" "rest" "of" "the"
[3,] "at" "the" "end" "of"
[4,] "to" "be" "abl" "to"
搜索并尝试了各种方法后,似乎do.call 和rbing 是解决方案。
words_table<-as.data.table(do.call(rbind,words_split))
但结果有 12 列,而不是 4 列:
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
[1,] "the" "end" "of" "the" "the" "end" "of" "the" "the" "end" "of" "the"
[2,] "the" "rest" "of" "the" "the" "rest" "of" "the" "the" "rest" "of" "the"
[3,] "at" "the" "end" "of" "at" "the" "end" "of" "at" "the" "end" "of"
[4,] "to" "be" "abl" "to" "to" "be" "abl" "to" "to" "be" "abl" "to"
[5,] "at" "the" "same" "time" "at" "the" "same" "time" "at" "the" "same" "time"
[6,] "in" "the" "middl" "of" "in" "the" "middl" "of" "in" "the" "middl" "of"
如果我对words_split 的一部分进行采样,比如前 4 个元素,然后做同样的事情,结果很好:
> words_head<-words_split[1:4]
> words_head
[[1]]
[1] "the" "end" "of" "the"
[[2]]
[1] "the" "rest" "of" "the"
[[3]]
[1] "at" "the" "end" "of"
[[4]]
[1] "to" "be" "abl" "to"
> class(words_head[1])
[1] "list"
> class(words_head[[1]])
[1] "character"
> words_head[[1]]
[1] "the" "end" "of" "the"
> words_head_comb<-do.call(rbind,words_head)
print(head(words_head_comb))
[,1] [,2] [,3] [,4]
[1,] "the" "end" "of" "the"
[2,] "the" "rest" "of" "the"
[3,] "at" "the" "end" "of"
[4,] "to" "be" "abl" "to"
为什么rbind() 会重复合并我的列表两次,当列表很大时,当列表很小时,它似乎工作?
【问题讨论】:
-
列表中的一个向量可能有 12 个元素而不是 4 个。rbinding 时,重复只有 4 个元素的行以达到 12 的大小。尝试
table(lengths(word_split))有了解列表中向量的长度。 -
返回 12 列时是否收到任何警告?你确定你的列表元素每个都有 4 个值吗?
-
感谢@Lamia 和@AntoniosK。我刚查了一下,发现有 124 个元素的长度从 5 到 12。我不知道这是怎么发生的,因为我使用了
quenteda包中的dfm函数来创建 4gram,所以我假设所有标记都是四个单词用破折号连接的。我猜有些词原本已经包括_。我使用while删除所有这些不规则元素,然后使用rbind。这次成功了。 -
FWIW:
do.call(rbind, words_split)创建一个矩阵,而不是 OP 要求的 data.frame。 -
@Uwe,是的,感谢您指出这一点——实际上我的实际代码中确实有
as.data.table包裹了do.call。我主要关心的是如何将所有列表组合成一个四列可搜索数据表以进行进一步处理。我现在将对其进行编辑。