有没有办法用已知格式修剪观察结果？答案

【问题标题】：Is there a way to trim out observations with a known format?有没有办法用已知格式修剪观察结果？
【发布时间】：2019-06-20 06:40:22
【问题描述】：

我正在使用一个包含 40 多个变量的数据库。每个案例都有其属性的唯一标识符。这些标识符中的一些已被输入到地址变量中。

标识符只能采用以下格式：

NA123456 - First letter constant - N, 1 Letter A-K, Numbers 1-9
SA123456 - First 2 letters constant - SA, 6 Numbers 0-9
MABC1234 - First letter constant - M, 3 Letters A-Z, 4 Numbers 0-9
QABC1234 - First letter constant - Q, 3 Letters A-Z, 4 Numbers 0-9
WABC1234 - First letter constant - W, 3 Letters A-Z, 4 Numbers 1-9
TABC1234 - First letter constant - T, 3 Letters A-Z, 4 Numbers 1-9
3ABCD123 - First number constant - 3, 3 Letters A-Z, 3 Numbers 1-9

我不确定如何在不创建查找表和使用 left_join 的情况下从地址文本中删除唯一标识符。查找表需要不断更新，使其非常麻烦。

我还没有找到这类事情的例子。我可能错过了一些东西。

我的数据如下所示：

Property                        Address               `Aa reference`
   <chr>                           <chr>                 <lgl>         
 1 PIC: 3WABG086                   260 SPRINGHURST ROAD  NA            
 2 PIC: 35PSR217                   1350 RIVER ROAD       NA            
 3 PIC# NH244157                   1038 QUONDONG ROAD    NA            
 4 PIC: 3GMUF425                   70 DIGBY ROAD         NA            
 5 PIC# 3GMUF425                   70 DIGBY ROAD         NA            
 6 PIC QTIWW0626                   REMOLEA               NA            
 7 PIC#EBWSE235                    BOX 191               NA            
 8 PIC #3WLKM019                   198 MONTGOMERY ROAD   NA            
 9 PIC # 3BWMM021                  149 ANDERSONS ROAD    NA            
10 PIC: 3WCGN034                   WERRIBEE              NA            
11 GARANGULA PIC: NH630488         PO BOX 84             NA            
12 GARANGULA PIC: NH630488         PO BOX 84             NA            
13 PIC: 3GMTL320                   2980 GLENELG HIGHWAY  NA            
14 GREENSLOPES PIC: MJKE0261       914 WEST KENTISH ROAD NA            
15 PIC: WFZB3246                   859 PFEIFFER ROAD     NA            
16 PIC: WFAY3549                   34605 ALBANY HIGHWAY  NA            
17 PIC: 3CEXK044                   2244 LAVERS HILL ROAD NA            
18 PIC: QGWW0462                   ELDERFIELD            NA            
19 PIC: 3WCGN034                   WERRIBEE              NA            
20 KAYA DORPER & WHITE DORPER STUD PIC: WABN0262         NA            
21 SPOTTSWOOD                      PIC QKDR0078          NA            
22 COOMBOONA HOLSTEINS             PIC 3SPSR217          NA            
23 ROSEVALE                        PIC: QKEV0169         NA            
24 NA                              PIC 3EGON009          NA            
25 NA                              PIC WFKPO316          NA            
26 IVADENE                         PIC 3WANP0T1          NA            
27 NA                              PIC ND225813          NA            
28 HEAVENLY VALLEY FARMS           PIC #NF538645         NA            
29 C/- CED WISE AB CENTRE          PIC: QCST0158         NA            
30 GARANGULA                       PIC # NH630488        NA

干净的数据最终会以 aa reference 列中的唯一标识符结束，并且不会用 NA 覆盖在正确变量中包含数据的观察结果。

非常感谢您的帮助。

【问题讨论】：

请通过添加dput(yourData[1:30, ]) 的输出来编辑您的问题。
ID 中有固定数量的字母或数字吗？例如。如果 Q 是有效的 ID，它是否总是后跟 3 个字母、4 个数字？如果是这样，那将更容易检查 ID。
你说数字 1-9.. 很多行都包含 nu,ber 0。应该排除它们吗？另外：搜索字符串的长度是固定的吗？
@Marius，是的，我已经更新了问题。
@Wimpel，我已更新问题以包含 0-9。

标签： r dplyr tidyr

【解决方案1】：

一个可能的答案，使用正则表达式和stringr::str_extract_all()

我假设您的数字应该是 0-9，而不是 1-9。如果不是，请将所有[0-9] 更改为[1-9]。
此外，如果您正在寻找特定数量（例如：n）的字母/数字重复，请将+ 更改为{n}，就像vec 中的第一个模式一样。

library( data.table )
library( stringr )

# NA123456 - First letter constant - N, Letter A-K, Numbers 1-9
# SA123456 - First 2 letters constant - SA, Numbers 1-9
# MABC1234 - First letter constant - M, Letters A-Z, Numbers 1-9
# QABC1234 - First letter constant - Q, Letters A-Z, Numbers 1-9
# WABC1234 - First letter constant - W, Letters A-Z, Numbers 1-9
# TABC1234 - First letter constant - T, Letters A-Z, Numbers 1-9
# 3ABCD123 - First number constant - 3, Letters A-Z, Numbers 1-9

#create a vector with all regex-patterns
#I assumed 1-9 should be 0-9 ??             <-- !!
vec <- c( "N[A-K]{1}[0-9]+", 
          "SA[0-9]+",
          "M[A-Z]+[0-9]+",
          "Q[A-Z]+[0-9]+",
          "W[A-Z]+[0-9]+",
          "T[A-Z]+[0-9]+",
          "3[A-Z]+[0-9]+" )
#paste patterns together to one large regex-OR-pattern
pattern <- paste( vec, collapse = "|" )
#extract all patterns from the column 'Property', and put (as vector) in Aa-reference
#extract all patterns from the column 'Property', and put (as vector) in Aa-reference
DT[, Aa_reference := stringr::str_extract_all( Address, pattern )]

输出

#                           Property               Address Aa_reference
# 1:                   PIC: 3WABG086  260 SPRINGHURST ROAD             
# 2:                   PIC: 35PSR217       1350 RIVER ROAD             
# 3:                   PIC# NH244157    1038 QUONDONG ROAD             
# 4:                   PIC: 3GMUF425         70 DIGBY ROAD             
# 5:                   PIC# 3GMUF425         70 DIGBY ROAD             
# 6:                   PIC QTIWW0626               REMOLEA             
# 7:                    PIC#EBWSE235               BOX 191             
# 8:                   PIC #3WLKM019   198 MONTGOMERY ROAD             
# 9:                  PIC # 3BWMM021    149 ANDERSONS ROAD             
# 10:                   PIC: 3WCGN034              WERRIBEE             
# 11:         GARANGULA PIC: NH630488             PO BOX 84             
# 12:         GARANGULA PIC: NH630488             PO BOX 84             
# 13:                   PIC: 3GMTL320  2980 GLENELG HIGHWAY             
# 14:       GREENSLOPES PIC: MJKE0261 914 WEST KENTISH ROAD             
# 15:                   PIC: WFZB3246     859 PFEIFFER ROAD             
# 16:                   PIC: WFAY3549  34605 ALBANY HIGHWAY             
# 17:                   PIC: 3CEXK044 2244 LAVERS HILL ROAD             
# 18:                   PIC: QGWW0462            ELDERFIELD             
# 19:                   PIC: 3WCGN034              WERRIBEE             
# 20: KAYA DORPER & WHITE DORPER STUD         PIC: WABN0262     WABN0262
# 21:                      SPOTTSWOOD          PIC QKDR0078     QKDR0078
# 22:             COOMBOONA HOLSTEINS          PIC 3SPSR217     3SPSR217
# 23:                        ROSEVALE         PIC: QKEV0169     QKEV0169
# 24:                            <NA>          PIC 3EGON009     3EGON009
# 25:                            <NA>          PIC WFKPO316     WFKPO316
# 26:                         IVADENE          PIC 3WANP0T1       3WANP0
# 27:                            <NA>          PIC ND225813     ND225813
# 28:           HEAVENLY VALLEY FARMS         PIC #NF538645     NF538645
# 29:          C/- CED WISE AB CENTRE         PIC: QCST0158     QCST0158
# 30:                       GARANGULA        PIC # NH630488     NH630488
#                            Property               Address Aa_reference

使用的样本数据

DT <- fread('
Property |                       Address |              Aa_reference
PIC: 3WABG086|                   260 SPRINGHURST ROAD|  NA            
PIC: 35PSR217|                   1350 RIVER ROAD      | NA            
PIC# NH244157|                   1038 QUONDONG ROAD    |NA            
PIC: 3GMUF425|                   70 DIGBY ROAD|         NA            
PIC# 3GMUF425|                   70 DIGBY ROAD |        NA            
PIC QTIWW0626 |                  REMOLEA        |       NA            
PIC#EBWSE235   |                 BOX 191         |      NA            
PIC #3WLKM019   |                198 MONTGOMERY ROAD|   NA            
PIC # 3BWMM021   |               149 ANDERSONS ROAD  |  NA            
PIC: 3WCGN034     |              WERRIBEE             | NA            
GARANGULA PIC: NH630488|         PO BOX 84             |NA            
GARANGULA PIC: NH630488 |        PO BOX 84|             NA            
PIC: 3GMTL320|                   2980 GLENELG HIGHWAY|  NA            
GREENSLOPES PIC: MJKE0261|       914 WEST KENTISH ROAD| NA            
PIC: WFZB3246           |        859 PFEIFFER ROAD|     NA            
PIC: WFAY3549|                   34605 ALBANY HIGHWAY|  NA            
PIC: 3CEXK044 |                  2244 LAVERS HILL ROAD| NA            
PIC: QGWW0462  |                 ELDERFIELD|            NA            
PIC: 3WCGN034   |                WERRIBEE|              NA            
KAYA DORPER & WHITE DORPER STUD| PIC: WABN0262|         NA            
SPOTTSWOOD|                      PIC QKDR0078  |        NA            
COOMBOONA HOLSTEINS|             PIC 3SPSR217   |       NA            
ROSEVALE            |            PIC: QKEV0169   |      NA            
NA|                              PIC 3EGON009     |     NA            
NA |                             PIC WFKPO316      |    NA            
IVADENE|                         PIC 3WANP0T1       |   NA            
NA      |                        PIC ND225813        |  NA            
HEAVENLY VALLEY FARMS|           PIC #NF538645        | NA            
C/- CED WISE AB CENTRE|          PIC: QCST0158         |NA            
GARANGULA|                       PIC # NH630488        |NA
', sep = "|")

【讨论】：

我认为问题要求从地址列中提取 ID（因为它们不应该真的存在，这似乎是清理数据过程的一部分）。
@Wimpel 感谢您迄今为止的帮助。如果我目前将我的数据框作为小标题，我是否必须将其转换为 data.table 或者我可以将其保留为 tibble 并更改代码？谢谢。
键是创建一个模式，然后使用str_extract_all()-函数。您可以在 data.table 上执行此操作（就像在我的示例中一样），但也可以使用您通常用来创建新列的任何其他方式。
@Wimpel，我在新列中得到了很多 character(0) 输出。当我尝试将 3 个新列组合在一起时，这些会覆盖 Aa reference 中的原始数据。我可以按层次进行组合吗？或者我最好在尝试unite之前删除character(0)？

【解决方案2】：

这最终奏效了：

vec <- c( "N[A-K]{1}[0-9]+", 
          "SA[0-9]+",
          "M[A-Z]+[0-9]+",
          "Q[A-Z]+[0-9]+",
          "W[A-Z]+[0-9]+",
          "T[A-Z]+[0-9]+",
          "3[A-Z]+[0-9]+" )

#paste patterns together to one large regex-OR-pattern
pattern <- paste( vec, collapse = "|" )

df <- df %>%
  mutate(`id1` = str_extract_all(`Property`, vec),
         `id2` = str_extract_all(`Address`, vec),
         `id1` = na_if(`Pic1`, "character(0)"),
         `id2` = na_if(`Pic2`, "character(0)")
  ) %>% 
  unite(id3, id1, id2, remove = TRUE, sep = " ") %>% 
  mutate(`id3` = str_extract_all(id3, vec),
         `id3` = na_if(`id3`, "character(0)"))

【讨论】：