如何通过 tf-idf 为 R 中的每个文档生成最热门的单词？答案

【问题标题】：How do I generate the top words by tf-idf for each document in R?如何通过 tf-idf 为 R 中的每个文档生成最热门的单词？
【发布时间】：2014-04-04 23:32:27
【问题描述】：

我从 R 中的 tm 包中获得了一个文档术语矩阵。

dd <- Corpus(VectorSource(train$text)) #Make a corpus object from a text vector
#Clean the text
dd <- tm_map(dd, stripWhitespace)
dd <- tm_map(dd, tolower)
dd <- tm_map(dd, removePunctuation)
dd <- tm_map(dd, removeWords, stopwords("english"))
dd <- tm_map(dd, stemDocument)
dd <- tm_map(dd, removeNumbers)
dtm <- DocumentTermMatrix(dd, control = list(weighting = weightTfIdf))

我找不到对文档术语矩阵进行操作以提取我想要的信息的方法：每个文档的 tf-idf 的前三个关键字。我怎么得到它？

编辑：示例文本（全部来自 Yelp Review 学术数据集）：

doc1 <- "Luckily, I didn't have to travel far to make my connecting flight. And for this, I thank you, Phoenix.  My brief layover was pleasant as the employees were kind and the flight was on time.  Hopefully, next time I can grace Phoenix with my presence for a little while longer."
doc2 <- "Nobuo shows his unique talents with everything on the menu. Carefully crafted features with much to drink. Start with the pork belly buns and a stout. Then go on until you can no longer."
doc3 <- "The oldish man who owns the store is as sweet as can be. Perhaps sweeter than the cookies or ice cream. Here's the lowdown: Giant ice cream cookie sandwiches for super cheap. The flavor permutations are basically endless. I had snickerdoodle with cookies and cream ice cream. It was marvelous."

我应该提一下，我有超过 180,000 个这种性质的文档，所以一个可扩展的解决方案，而不是仅适用于这些特定示例的解决方案，会很棒。

【问题讨论】：

我将添加一些示例文本。按任意顺序列出并列的三个 - 我不认为这是一个非常普遍的情况，但我明白你的意思。
例如apply(as.data.frame(as.matrix(dtm)), 1, function(x) tail(names(sort(x)), 3)) - 但是，领带问题仍然存在，@TylerRinker 提到了
很好的解决方案。应用功能如何在数据框上工作？无法从文档中找到详细说明。

标签： r text tf-idf tm

【解决方案1】：

这行得通：

apply(dtm, 1, function(x) {
    x2 <- sort(x, TRUE)
    x2[x2 >= x2[3]]
})

## $doc1
##   flight  phoenix     time 
## 0.126797 0.126797 0.126797 
## 
## $doc2
##      belli        bun       care      craft      drink    everyth     featur 
## 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 
##       menu       much      nobuo       pork       show      start      stout 
## 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 0.08805347 
##     talent      uniqu 
## 0.08805347 0.08805347 
## 
## $doc3
##     cream     cooki       ice 
## 0.2113283 0.1584963 0.1584963

如果您希望它扩大规模，我会使用并行计算。

【讨论】：