从语料库到 R 的 tm 中的单个 .txt 文件

Question

我有一个包含 6000 行和 2 列的 .csv 文件。我想将每一行写为一个单独的文本文件。关于如何在 tm 中完成此操作的任何想法？我尝试了writeCorpus()，但该函数只吐出 150 个 .txt 文件而不是 6000 个。这是内存问题还是我对代码做错了什么？

 library(tm)
 revs<-read.csv("dprpfinals.csv",header=TRUE)
 corp<-Corpus(VectorSource(revs$Review))
 writeCorpus(corp,path=".",filenames=paste(seq_along(revs),".txt",sep=""))

Answer 1

这是一个将文本拆分为段落、删除空行并将这些行写入文本文件的示例。然后你需要处理文本文件。

txt="Argument split will be coerced to character, so you will see uses with split = NULL to mean split = character(0), including in the examples below.

Note that splitting into single characters can be done via split = character(0) ; the two are equivalent. The definition of 'character’ here depends on the locale: in a single-byte locale it is a byte, and in a multi-byte locale it is the unit represented by a ‘wide character’ (almost always a Unicode code point).

A missing value of split does not split the corresponding element(s) of x at all."

txt2<-data.frame(para = strsplit(txt, "\n")[[1]],stringsAsFactors=FALSE)
txt3<-txt2[txt2$para!="",]

npara = length(txt3)
for (ip in seq(1,npara)) {
  fname = paste("paragraph_",ip,".txt",sep="")
  fileConn<-file(fname)
  writeLines(txt3[ip], fileConn)
  close(fileConn)  
}

Answer 2

对此无需使用tm，这是一个可重现的示例，它制作了一个包含 6000 行和两列的 CSV 文件，将其读入，然后将其转换为 6000 个 txt 文件

先为示例准备一些数据...

# from http://hipsum.co/?paras=4&type=hipster-centric
txt <- "Brunch single-origin coffee photo booth, meggings fixie stumptown pickled mumblecore slow-carb aesthetic ennui Odd Future blog plaid Bushwick. Seitan keffiyeh hashtag Portland, kitsch irony authentic vegan post-ironic. Actually pop-up flexitarian kale chips ethical authentic, stumptown meggings. Photo booth Helvetica farm-to-table Neutra. Selfies blog swag, lomo viral meh chillwave distillery deep v Truffaut. Squid Cosby sweater irony, art party mustache Vice Wes Anderson Bushwick McSweeney's locavore roof party paleo. 3 wolf moon salvia gentrify, taxidermy street art banh mi Portland deep v small batch Truffaut."

# get n random samples of this paragraph
n <- 6000
txt_split <- unlist(strsplit(txt, split = " "))
txts <- sapply(1:n, function(i) paste(sample(txt_split, 10, replace = TRUE), 
                                             collapse  = " "))

# make dataframe then CSV file, two cols, n rows.
my_csv <- data.frame( col_one = 1:n,
                      col_two = txts)
write.csv(my_csv, "my_csv.csv", row.names = FALSE, quote = TRUE)

现在我们有一个 CSV 文件，可能与您的文件类似，我们可以将其读入：

# Read in the CSV file...

x <- read.csv("my_csv.csv", header = TRUE, stringsAsFactors = FALSE)

现在我们可以将 CSV 文件的每一行写入一个单独的文本文件（它们将出现在您的工作目录中）：

# Write each row of the CSV to a txt file
sapply(1:nrow(x), function(i) write.table(paste(x[i,], collapse = " "), 
                                          paste0("my_txt_", i, ".txt"), 
                                          col.names = FALSE, row.names = FALSE))

如果你真的想使用tm，那你就在正确的轨道上，这对我来说很好：

# Read in the CSV file...
x <- read.csv("my_csv.csv", header = TRUE, stringsAsFactors = FALSE)
library(tm)
my_corpus <- Corpus(DataframeSource(x))
writeCorpus(my_corpus)

更接近你的例子对我来说也很好：

corp <- Corpus(VectorSource(x$col_one))
writeCorpus(corp)

如果它不适合您，则可能是您的 CSV 文件出现了一些异常情况、一些奇怪的字符等等。如果没有关于您的具体问题的更多细节，很难说。