【问题标题】:Update DataFrame Based on Grouped Values from a different DataFrame根据来自不同 DataFrame 的分组值更新 DataFrame
【发布时间】:2017-02-26 09:12:10
【问题描述】:

我有一个数据框,它的列需要根据查找表进行更新。查找表基于分组集。如果未找到匹配项,则要更新的值将留空。

这是我的输入数据:

dput(DF_Generated)
structure(list(PO_ID = c("P1234", "P1234", "P1234", "P1234", 
"P1234", "P1234", "P1234", "P2345", "P2345", "P2345", "P3456", 
"P3456", NA, NA), SO_ID = c("S1", "S1", "S1", "S2", "S2", "S2", 
"S3", "S3", "S4", "S5", "S7", NA, "S10", "S10"), F_Year = c(2012, 
2012, 2012, 2013, 2013, 2013, 2013, 2011, 2011, 2012, 2014, 2014, 
2015, 2015), Product_ID = c("385X", "385X", "385X", "450X", "450X", 
"900X", "N9X", "3700", "3700", "3800", "A11U", "385X", "2700", 
"3700"), Revenue = c(16.6666666666667, 16.6666666666667, 16.6666666666667, 
35, 35, 35, 100, 100, -50, 20, 50, 20, 100, 10), Quantity = c(1, 
1, 1, 10, 10, 20, 20, 20, -10, 20, 20, 5, 40, 20), Location1 = c("MA", 
"NY", "WA", "NY", "WA", "NY", NA, "IL", "IL", NA, "MN", NA, "CA", 
NA), Name = c("N1", "N1", "N1", "N1", "N1", "N1", NA, "N2", "N2", 
NA, "N3", NA, "N4", NA)), class = c("tbl_df", "tbl", "data.frame"
), row.names = c(NA, -14L), .Names = c("PO_ID", "SO_ID", "F_Year", 
"Product_ID", "Revenue", "Quantity", "Location1", "Name"))

这是我的查找表:

dput(DF_Lookup_2)
structure(list(PO_ID = c("P1234", "P1234", "P1234", "P1234", 
"P1234", "P2345", "P2345", "P3456", NA), SO_ID = c("S1", "S1", 
"S1", "S2", "S2", "S3", "S4", "S7", "S10"), F_Year = c(2012, 
2012, 2012, 2013, 2013, 2011, 2011, 2014, 2015), Location1 = c("MA", 
"NY", "WA", "NY", "WA", "IL", "IL", "MN", "CA"), Name = c("N1", 
"N1", "N1", "N1", "N1", "N2", "N2", "N3", "N4")), class = c("tbl_df", 
"tbl", "data.frame"), row.names = c(NA, -9L), .Names = c("PO_ID", 
"SO_ID", "F_Year", "Location1", "Name"))

预期的输出是:

dput(DFO)
structure(list(PO_ID = c("P1234", "P1234", "P1234", "P1234", 
"P1234", "P1234", "P1234", "P2345", "P2345", "P2345", "P3456", 
"P3456", NA, NA), SO_ID = c("S1", "S1", "S1", "S2", "S2", "S2", 
"S3", "S3", "S4", "S5", "S7", NA, "S10", "S10"), F_Year = c(2012, 
2012, 2012, 2013, 2013, 2013, 2013, 2011, 2011, 2012, 2014, 2014, 
2015, 2015), Product_ID = c("385X", "385X", "385X", "450X", "450X", 
"900X", "N9X", "3700", "3700", "3800", "A11U", "385X", "2700", 
"3700"), Revenue = c(16.6666666666667, 16.6666666666667, 16.6666666666667, 
35, 35, 35, 100, 100, -50, 20, 50, 20, 100, 10), Quantity = c(1, 
1, 1, 10, 10, 20, 20, 20, -10, 20, 20, 5, 40, 20), Location1 = c("MA", 
"NY", "WA", "NY", "WA", "NY", NA, "IL", "IL", NA, "MN", NA, "CA", 
"CA"), Name = c("N1", "N1", "N1", "N1", "N1", "N1", NA, "N2", 
"N2", NA, "N3", NA, "N4", "N4")), .Names = c("PO_ID", "SO_ID", 
"F_Year", "Product_ID", "Revenue", "Quantity", "Location1", "Name"
), row.names = c(NA, 14L), class = "data.frame")

逻辑:

查找是基于三列完成的:PO_ID,SO_ID,F_Year。如果找到匹配项,则仅当数据框需要更新时才会覆盖条目。例如,列Location1Name 的行使得PO_ID = P1234SO_ID = S1F_Year = 2012 不应被覆盖,因为它们的值已经存在于查找中桌子。但是,Location1Name 列中的条目对于这样的行:PO_ID = NASO_ID = S10F_Year = 2015 需要使用来自的有效值进行更新查找表,分别为CAN4。我尝试使用Data.Table,但我下面的代码覆盖了所有条目,这是不正确的。

我阅读了Compare and merge two dataframes 线程,并尝试这样做,但代码覆盖了我不需要查找的现有条目。

这是我的代码:

data.table::setDT(DF_Generated)
data.table::setDT(DF_Lookup_2)
data.table::setkey(DF_Generated,PO_ID,SO_ID,F_Year)
data.table::setkey(DF_Lookup_2,PO_ID,SO_ID,F_Year)

DF_Generated[DF_Lookup_2,on=c("PO_ID","SO_ID","F_Year"),c("Location1","Name"):=list(i.Location1,i.Name)]

我有两个问题:

问题:1) 我使用Data.Table 因为我的实际数据很大。所以,我正在寻找基于data.table 的解决方案。如何修复我的 data.table 代码?

问题 2) 如果这是推荐的方式,我也愿意接受 dplyr

但是,就我的学习而言,如果您能帮助我解决这两种解决方案,我将不胜感激。我是初学者,还在学习这两个包。

【问题讨论】:

  • @akrun - 感谢您的帮助。如果你运行上面的代码,你会看到Location1 将全部是WA(不同于原始数据集DF_Generated。请查看Location1Name 列中的预期输出。这有帮助吗?也许我没有听懂你的问题。

标签: r data.table dplyr


【解决方案1】:

这样做的另一个方法是(按条件)加入DF_Lookup_2,然后分配回DF_Generated。反过来这样做的原因是因为在和X[Y] 连接中,结果连接的长度为Y,因此DF_Lookup_2[DF_Generated] 类型的连接将为我们提供所需的长度。然后,我们可以将它原样放回DF_Generated

DF_Generated[is.na(Location1) | is.na(Name), c("Location1", "Name"):= 
               DF_Lookup_2[DF_Generated[is.na(Location1) | is.na(Name)], 
                           .(Location1, Name), on = .(PO_ID, SO_ID, F_Year)]]


identical(DF_Generated, setDT(DFO))
## [1] TRUE

【讨论】:

    【解决方案2】:

    我们可以在加入on'PO_ID'、'SO_ID'和'F_Year'时创建两个新列,然后使用set更新有NA的旧列

    setDT(DF_Generated)[setDT(DF_Lookup_2), c("Location1N", "NameN") := list(i.Location1, 
                                   i.Name),on = .(PO_ID, SO_ID, F_Year)]
    nm1 <- c("Location1", "Name")
    nm2 <- paste0(nm1, "N")
    for(j in seq_along(nm1)){
     set(DF_Generated, i = which(is.na(DF_Generated[[nm1[j]]])), j=nm1[j], 
                         value = DF_Generated[[nm2[j]]][is.na(df1[[nm1[j]]])])
    }
    DF_Generated[, (nm2) := NULL][]
    
    identical(setDT(DFO), DF_Generated)
    #[1] TRUE
    

    【讨论】:

    • 非常感谢您的帮助。如果您不介意,我有两个后续问题:a) 您认为dplyr 会更简单吗? b) 你认为这段代码的性能会好吗?我问这个是因为在我的原始数据集中,我有大约 1 亿行和约 28 列。我还没有在原始数据集上运行你的代码,因为我需要时间来定制上面的代码。我是初学者,没有太多的线索。因此,我很感激你的想法。
    • @watchtower tidyverse 中有一个 coalesce 函数,但如果有重复则 left_join 将不起作用
    • 非常感谢。非常尊重,我收到错误,不知道为什么。 "Error in data.table::set(DF_Generated, i =which(is.na(DF_Generated[[nm1[j]]])), : RHS of assignment to existing column 'Location1' is zero length but not NULL. If you intend to delete the column use NULL. Otherwise, the RHS must have length &gt; 0; e.g., NA_integer_. If you are trying to change the column type to be an empty list column then, as with all column type changes, provide a full length RHS vector such as vector('list',nrow(DT)); i.e., 'plonk' in the new column. 知道为什么会发生这种情况吗?
    • @watchtower 这是基于您帖子中的示例还是原始数据集。会不会是包版本的原因?我正在使用 data.table_1.10.0
    • @watchtower 如果在新版本中有问题,请继续使用 DavidArenburg 的代码
    猜你喜欢
    • 2021-08-12
    • 2022-01-23
    • 1970-01-01
    • 1970-01-01
    • 2022-01-09
    • 2016-09-06
    • 1970-01-01
    • 2021-07-12
    • 1970-01-01
    相关资源
    最近更新 更多