R - 删除重复字符之间的所有换行符

Question

我目前正在进行情绪分析的数据清理工作，并且正在使用数据框形式的大型新闻文章数据集。我需要能够分析数据框的每行一篇文章，并且正在寻找一种方法来删除第一个“======”和第二个“======”之间的换行符，重复贯穿整个数据框。此外，在内容“折叠到自身”之后，我希望保留发布者和日期列。

df <-  matrix(c("======","NA","NA","Daily Bugle Dec 31","Daily Bugle", "Dec 31" ,"Wookies are","NA","NA",". recreationally", "NA","NA", "using drugs at a", "NA", "NA", "higher rate than", "NA", "NA","ever before.", "NA", "NA","======", "NA", "NA" ),ncol=3,byrow=TRUE)
colnames(df) <- c("content","publisher","date")
df <- as.data.frame(df)
df[ df == "NA" ] <- NA

给出这个：

content              publisher   date
======               <NA>         <NA>
Daily Bugle, Dec 31  Daily Bugle Dec 31
Wookies are          <NA>         <NA>
recreationally       <NA>         <NA>
using drugs at a     <NA>         <NA>
higher rate than     <NA>         <NA>
ever before.         <NA>         <NA>
======               <NA>         <NA>

我想要这样的东西：

content                                           publisher     date
======
Wookies are recreationally using drugs at a hig... Daily Bugle Dec 31           
======
Article 2
======
Article 3
======

希望这很清楚。我对 R 比较陌生。

Answer 1

• 每篇文章都以'===' 开头，因此可以用作文章编号。
• 删除每篇文章的content 的第一个值。
• 保留publisher 和date 的第一个值。

library(dplyr)

df %>%
  mutate(article_no = cumsum(grepl('===', content))) %>%
  filter(!grepl('===', content)) %>%
  group_by(article_no) %>%
  summarise(content = paste0(content[-1], collapse = ''), 
            publisher = publisher[1], 
            date = date[1])

#  article_no content                                                                 publisher   date  
#       <int> <chr>                                                                   <chr>       <chr> 
#1          1 Wookies are. recreationallyusing drugs at ahigher rate thanever before. Daily Bugle Dec 31

Answer 2

为了帮助你，首先我需要准备一些数据。

library(tidyverse)
articles = read.table(
  header = TRUE,sep = ",",text="
content,publisher,date
======,NA,NA
Daily News Dec 27,Daily News,Dec 27
Wookies are,NA,NA
. recreationally,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
ever before.,NA,NA
ever before.,NA,NA
ever before.,NA,NA
ever before.,NA,NA
======,NA,NA
======,NA,NA
Daily News Dec 28,Daily News,Dec 28
Wookies are,NA,NA
. recreationally,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
ever before.,NA,NA
ever before.,NA,NA
ever before.,NA,NA
ever before.,NA,NA
======,NA,NA
======,NA,NA
Daily News Dec 30,Daily News,Dec 30
Wookies are,NA,NA
. recreationally,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
ever before.,NA,NA
ever before.,NA,NA
======,NA,NA
======,NA,NA
Daily Bugle Dec 31,Daily Bugle,Dec 31
Wookies are,NA,NA
. recreationally,NA,NA
using drugs at a,NA,NA
higher rate than,NA,NA
ever before.,NA,NA
======,NA,NA
======,NA,NA
Weekly News Dec 31,Weekly News,Dec 31
Wookies are,NA,NA
. recreationally,NA,NA
higher rate than,NA,NA
ever before.,NA,NA
======,NA,NA") %>%
  as_tibble() %>% 
  mutate(publisher = ifelse(publisher=="NA", NA, publisher),
         date = ifelse(date=="NA", NA, date))
articles

输出

# A tibble: 52 x 3
   content           publisher  date  
   <chr>             <chr>      <chr> 
 1 ======            NA         NA    
 2 Daily News Dec 27 Daily News Dec 27
 3 Wookies are       NA         NA    
 4 . recreationally  NA         NA    
 5 using drugs at a  NA         NA    
 6 higher rate than  NA         NA    
 7 using drugs at a  NA         NA    
 8 higher rate than  NA         NA    
 9 using drugs at a  NA         NA    
10 higher rate than  NA         NA    
# ... with 42 more rows

我希望这是您的数据格式。对我来说，这是五篇文章。

现在让我们添加一个转换函数和一个简单的突变。

fConvert = function(data) tibble(
  publisher = data$publisher[2],
  date = data$date[2],
  content = data %>% slice(3:(nrow(.)-1)) %>% 
    pull(content) %>% paste(collapse = " ")
)

articles %>% mutate(
  idArticle = ifelse(!is.na(publisher),1, 0) %>% 
    cumsum() %>% lead(default=.[length(.)]) 
) %>% group_by(idArticle) %>% 
  nest() %>% 
  group_modify(~fConvert(.x$data[[1]]))

输出

# A tibble: 5 x 4
# Groups:   idArticle [5]
  idArticle publisher   date   content                                                                                            
      <dbl> <chr>       <chr>  <chr>                                                                                              
1         1 Daily News  Dec 27 Wookies are . recreationally using drugs at a higher rate than using drugs at a higher rate than u~
2         2 Daily News  Dec 28 Wookies are . recreationally using drugs at a higher rate than ever before. ever before. ever befo~
3         3 Daily News  Dec 30 Wookies are . recreationally using drugs at a higher rate than ever before. ever before.           
4         4 Daily Bugle Dec 31 Wookies are . recreationally using drugs at a higher rate than ever before.                        
5         5 Weekly News Dec 31 Wookies are . recreationally higher rate than ever before.     

如您所见，我能够提取五篇文章，尽管它们的长度不同，并将所有行合并成一篇content。希望这就是你的意思。