如何编写循环以逐行处理不规则/非矩形数据

Question

如果这看起来很简单，我深表歉意，但是我花了很多时间在 Google 上兜圈子搜索，但我似乎找不到这个问题的答案...

我有一系列需要导入的 excel 文件（.csv 类型）。它们由由 | 分隔的 9 列组成的数据组成。看起来像这样：

1  CustID|Animal Adopted|Description|Year|Adoption date|dob|Fee|Paid        
2  123|Horse|I got a Horse|2011|2011-01-31|1991-03-01|8.00|NO       
3 0101|Horse|I got a Horse|2006|2006-01-31|1987-21-01|50.00|NO      
4 1929|Fish|I adopted a fish    and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
5 199|Fish|I adopted a fish and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
6 0101|Dog|another really long description  for adopting a dog because they are awesome 
7 |2003|2003-05-11|1999-04-01|50.00|(null)      
8 0102|Dog|another really long description  for adopting a dog because they are awesome 
9 |2003|2003-05-11|1989-04-01|51.00|(null)      
10 9330|Cat|Mr.Kitty is a kitty who is so fat   and cute    and furry|2010|2010-10-11|1986-10-20|10000.00|YES
11 1933|Hamster|Doing a dance   in tube city.   
12 |2011|2011-12-05|2005-06-01|4.00|(null)  

因此，当导入到 R 中时，它会这样做

    CustID Animal.Adopted                                                                  Description       Year Adoption.date        dob   Fee Paid..
1     123          Horse                                                                I got a Horse       2011    2011-01-31 1991-03-01     8   NO,,
2     101          Horse                                                                I got a Horse       2006    2006-01-31 1987-21-01    50   NO,,
3    1929           Fish  I adopted a fish,and for some reason,this requires a very long description        2001    2001-12-31 1991-10-01     5    YES
4     199           Fish  I adopted a fish,and for some reason,this requires a very long description        2001    2001-12-31 1991-10-01     5    YES
5     101            Dog another really long description,for adopting a dog because they are awesome,                                        NA       
6      NA           2003                                                                   2003-05-11 1999-04-01         50.00   (null),,    NA       
7     102            Dog another really long description,for adopting a dog because they are awesome,                                        NA       
8      NA           2003                                                                   2003-05-11 1989-04-01         51.00   (null),,    NA       
9    9330            Cat                         Mr.Kitty is a kitty who is so fat,and cute,and furry       2010    2010-10-11 1986-10-20 10000    YES
10   1933        Hamster                                                 Doing a dance,in tube city.,                                        NA       
11     NA           2011                                                                   2011-12-05 2005-06-01          4.00   (null),,    NA       
> 

这会严重影响我以后尝试对数据进行分类时结果的准确性。第 5 行和第 6 行、第 7 行和第 8 行以及第 10 行和第 11 行代表了问题，因为列数据的一半在第一行，另一半在第二行。

我在几年前发现了这个问题@Layla，关于逐行阅读文本文件，并试图将接受的答案与个性化的 for 循环混合。但是，我不明白为什么 while 循环条件为 TRUE...

我想知道是否有一种方法可以对其进行编码，以便每次 r 遇到只有三列的行时，它会向上移动下一行以填充最后两列。但我只希望在完全匹配的数据中出现这种情况，因为我可能会丢失整个数据中的随机名称或位置，因为我不想像前面提到的那样与整个数据集不匹配。

我不知道如何指示循环查找丢失的数据。我尝试了类似if line(ncol = 3) {Merge()} 的方法，但老实说，这没有任何意义......

我已经尝试过readLines() ... 但它仍然以完全相同的方式读取所有内容，因此我认为这需要与其他代码结合使用。而我所有的 Google 搜索都只是出现相同的基本“readLines 一次读取一行中的连接”，然后就是这样。

我觉得我有这些碎片，但我仍然不知道如何将它们组合在一起。

作为警告，我对此很陌生。即使是解释如何编写手动读取代码的例程的有用文章也会有所帮助。关于如何使用read.csv, read.tsv, read.xlsx 等读取文件的信息很多，但关于如何手动处理读取数据的信息却很少。

Answer 1

我找到了“如何将带有换行符的数据从文本文件导入 R？”的答案。有帮助。但首先我认为更改“|”可能会有所帮助逗号分隔符。

txt <- "1  CustID|Animal Adopted|Description|Year|Adoption date|dob|Fee|Paid        
2  123|Horse|I got a Horse|2011|2011-01-31|1991-03-01|8.00|NO       
3 0101|Horse|I got a Horse|2006|2006-01-31|1987-21-01|50.00|NO      
4 1929|Fish|I adopted a fish    and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
5 199|Fish|I adopted a fish and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
6 0101|Dog|another really long description  for adopting a dog because they are awesome 
7 |2003|2003-05-11|1999-04-01|50.00|(null)      
8 0102|Dog|another really long description  for adopting a dog because they are awesome 
9 |2003|2003-05-11|1989-04-01|51.00|(null)      
10 9330|Cat|Mr.Kitty is a kitty who is so fat   and cute    and furry|2010|2010-10-11|1986-10-20|10000.00|YES
11 1933|Hamster|Doing a dance   in tube city.   
12 |2011|2011-12-05|2005-06-01|4.00|(null)
"

编辑：

我意识到将其视为文本并仅将 "nn<spc>|" 后面的 EOL 字符替换为

 txt2 <- gsub("\\n\\d{1,4} {1,3}\\|", "|", txt)

获取常规结果：

 count.fields( textConnection(txt2),sep="|")
[1] 8 8 8 8 8 8 8 8 8

现在用 read.table 读取文件的其余部分。

> read.table( text=txt2,sep="|", header=TRUE)
  X1..CustID Animal.Adopted
1     2  123          Horse
2     3 0101          Horse
3     4 1929           Fish
4      5 199           Fish
5     6 0101            Dog
6     8 0102            Dog
7    10 9330            Cat
8    11 1933        Hamster
                                                                     Description Year Adoption.date
1                                                                  I got a Horse 2011    2011-01-31
2                                                                  I got a Horse 2006    2006-01-31
3 I adopted a fish    and for some reason this requires a very long description  2001    2001-12-31
4    I adopted a fish and for some reason this requires a very long description  2001    2001-12-31
5  another really long description  for adopting a dog because they are awesome  2003    2003-05-11
6  another really long description  for adopting a dog because they are awesome  2003    2003-05-11
7                      Mr.Kitty is a kitty who is so fat   and cute    and furry 2010    2010-10-11
8                                               Doing a dance   in tube city.    2011    2011-12-05
         dob   Fee         Paid
1 1991-03-01     8    NO       
2 1987-21-01    50     NO      
3 1991-10-01     5          YES
4 1991-10-01     5          YES
5 1999-04-01    50 (null)      
6 1989-04-01    51 (null)      
7 1986-10-20 10000          YES
8 2005-06-01     4       (null)
> 

Answer 2

在这里，我假设行号实际上不是数据集的一部分，因为您已经正确地读取了它，所以我先gsub 它。请参阅我的数据输入的答案结尾。

我注意到两行中的第二行总是以分隔符| 开头；这样可以轻松删除之前的换行符，然后可以正常读入。我在意识到之前尝试过的其他选项如下。

> txt3 <- gsub("\n|", "|", txt, fixed=TRUE)
> read.table(text=txt3, sep="|", header=TRUE)
  CustID Animal.Adopted                                                                    Description Year Adoption.date        dob   Fee   Paid
1    123          Horse                                                                  I got a Horse 2011    2011-01-31 1991-03-01     8     NO
2    101          Horse                                                                  I got a Horse 2006    2006-01-31 1987-21-01    50     NO
3   1929           Fish I adopted a fish    and for some reason this requires a very long description  2001    2001-12-31 1991-10-01     5    YES
4    199           Fish    I adopted a fish and for some reason this requires a very long description  2001    2001-12-31 1991-10-01     5    YES
5    101            Dog  another really long description  for adopting a dog because they are awesome  2003    2003-05-11 1999-04-01    50 (null)
6    102            Dog  another really long description  for adopting a dog because they are awesome  2003    2003-05-11 1989-04-01    51 (null)
7   9330            Cat                      Mr.Kitty is a kitty who is so fat   and cute    and furry 2010    2010-10-11 1986-10-20 10000    YES
8   1933        Hamster                                               Doing a dance   in tube city.    2011    2011-12-05 2005-06-01     4 (null)

因为两行中的第二行始终以分隔符| 开头，所以CustID 列在您读入时为NA。您可以使用它来提前移动这些行。另请注意，现在所有字段都是字符，而不是数字，因为最初读入时，这些字段中有字符；您必须在阅读后转换这些内容。

d <- read.table(text=txt, sep="|", fill=NA, header=TRUE, as.is=TRUE)
k <- which(is.na(d$CustID))
d[k-1,4:8] <- d[k,2:6]
d[-k,]

或者，如果行之间的间隔可能并不总是在同一列，我们可以将其读入，拆分为单元，然后制作矩阵。

txt2 <- txt
txt2 <- gsub("\n", "|", txt2, fixed=TRUE)
txt2 <- gsub("||", "|", txt2, fixed=TRUE)
txt2 <- strsplit(txt2, "|", fixed=TRUE)[[1]]
d2 <- matrix(txt2, ncol=8, byrow=TRUE)
colnames(d2) <- d2[1,]
d2 <- as.data.frame(d2[-1,], as.is=TRUE)

数据输入：

txt <- "1  CustID|Animal Adopted|Description|Year|Adoption date|dob|Fee|Paid        
2  123|Horse|I got a Horse|2011|2011-01-31|1991-03-01|8.00|NO
3 0101|Horse|I got a Horse|2006|2006-01-31|1987-21-01|50.00|NO
4 1929|Fish|I adopted a fish    and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
5 199|Fish|I adopted a fish and for some reason this requires a very long description |2001|2001-12-31|1991-10-01|5.00|YES
6 0101|Dog|another really long description  for adopting a dog because they are awesome 
7 |2003|2003-05-11|1999-04-01|50.00|(null)
8 0102|Dog|another really long description  for adopting a dog because they are awesome 
9 |2003|2003-05-11|1989-04-01|51.00|(null)
10 9330|Cat|Mr.Kitty is a kitty who is so fat   and cute    and furry|2010|2010-10-11|1986-10-20|10000.00|YES
11 1933|Hamster|Doing a dance   in tube city.   
12 |2011|2011-12-05|2005-06-01|4.00|(null)
"
txt <- sub("1 *", "", txt)
txt <- gsub("\n[0-9]+ *", "\n", txt)