【发布时间】:2020-10-06 08:33:42
【问题描述】:
我有一个循环遍历多年数据的脚本,一次一年。每一年的数据都由多个数据框组成,这些数据框位于名为all_input 的列表中。在循环开始时(读入数据后),我试图在其余处理之前以相同格式获取所有年份的数据。
我遇到的问题是列名不统一。
我要保留的每个数据框中包含 5 列,我希望它们被称为 total_emissions uom tribal_name st_usps_cd 和 description。在某些数据框中,它们已经具有这些名称,而在其他数据框中,它们具有各种名称,例如 pollutant.desc 或 pollutant_desc。
我目前的做法是这样的:
# Create a mapping file for the column names
header_map <- data.frame(orignal_col = c( "pollutant_desc", "pollutant.desc", "emissions.uom", "total.emissions", "tribal.name", "state" ),
new_col = c( "description", "description", "uom", "total_emissions", "tribal_name", "st_usps_cd" ), stringsAsFactors = FALSE)
# change the column names
lapply(all_input, function(x) {
names(x)[match(header_map$orignal_col, names(x))] <- header_map$new_col
x
}) -> all_input
这会创建一个如下所示的标头映射文件:
original_col new_col
pollutant_desc description
pollutant.desc description
emissions.uom uom
total.emissions total_emissions
tribal.name tribal_name
state st_usps_cd
我得到的错误如下:
Error in names(x)[match(header_map$orignal_col, names(x))] <- header_map$new_col :
NAs are not allowed in subscripted assignments
我知道,由于处理具有不同列名的新年份数据,我必须手动将条目添加到头文件中,但是我怎样才能让它工作呢?
假样本数据。 df1 和 df2 代表“2017”数据的格式,其中多列需要更改名称,但当前名称在数据帧之间保持一致。 df3 表示“2011”数据,其中所有列名都应如此。 df4 代表“2014”数据,其中唯一需要更改的列是pollutant_desc。请注意,每个数据框中都有多余的列不需要并且可以忽略。提醒一下,这些数据帧并不是同时读取的。循环是按年份进行的,因此将格式化和处理 df1 和 df2(在列表 all_input 中)。然后删除所有数据,并使用未来几年的数据框创建一个新的all_input 列表,这些数据框将具有不同的列名。代码必须在不改变的情况下使用多年。
> dput(df1)
structure(list(total.emissions = structure(1:2, .Label = c("100",
"300"), class = "factor"), emissions.uom = structure(1:2, .Label = c("LB",
"TON"), class = "factor"), international = c(TRUE, TRUE), hours = structure(2:1, .Label = c("17",
"3"), class = "factor"), tribal.name = structure(2:1, .Label = c("FLLK",
"SUWJG"), class = "factor"), state = structure(1:2, .Label = c("AK",
"MN"), class = "factor"), pollutant.desc = structure(1:2, .Label = c("Methane",
"NO2"), class = "factor"), policy = c(TRUE, FALSE)), class = "data.frame", row.names = c(NA,
-2L))
> dput(df2)
structure(list(total.emissions = structure(2:1, .Label = c("20",
"400"), class = "factor"), emissions.uom = structure(c(1L, 1L
), .Label = "TON", class = "factor"), international = c(FALSE,
TRUE), hours = structure(2:1, .Label = c("1", "8"), class = "factor"),
tribal.name = structure(2:1, .Label = c("SOSD", "WMFJU"), class = "factor"),
state = structure(2:1, .Label = c("SD", "WY"), class = "factor"),
pollutant.desc = structure(1:2, .Label = c("CO2", "SO2"), class = "factor"),
policy = c(FALSE, FALSE)), class = "data.frame", row.names = c(NA,
-2L))
> dput(df3)
structure(list(total_emissions = structure(2:1, .Label = c("200",
"30"), class = "factor"), uom = structure(c(1L, 1L), .Label = "TON", class = "factor"),
boundaries = structure(2:1, .Label = c("N", "Y"), class = "factor"),
tribal_name = structure(2:1, .Label = c("SOSD", "WMFJU"), class = "factor"),
st_usps_cd = structure(2:1, .Label = c("ID", "KS"), class = "factor"),
description = structure(c(1L, 1L), .Label = "SO2", class = "factor"),
policy = c(FALSE, TRUE), time = structure(1:2, .Label = c("17",
"7"), class = "factor")), class = "data.frame", row.names = c(NA,
-2L))
> dput(df4)
structure(list(total_emissions = structure(2:1, .Label = c("700",
"75"), class = "factor"), uom = structure(c(1L, 1L), .Label = "LB", class = "factor"),
tribal_name = structure(1:2, .Label = c("SSJY", "WNCOPS"), class = "factor"),
st_usps_cd = structure(1:2, .Label = c("MO", "NY"), class = "factor"),
pollutant_desc = structure(2:1, .Label = c("CO2", "Methane"
), class = "factor"), boundaries = structure(c(1L, 1L), .Label = "N", class = "factor"),
policy = c(FALSE, FALSE), time = structure(1:2, .Label = c("2",
"3"), class = "factor")), class = "data.frame", row.names = c(NA,
-2L))
谢谢!
【问题讨论】:
-
match在未找到其参数之一时返回NA。找到没有找到的列,你会很好。 -
@r2evans 某些列未找到,因为它们并非每年都存在。例如,“pollutant_desc”用于 2014 年的数据,但“pollutant.desc”用于 2017 年的数据。这是我试图克服的问题
-
查看样本数据有助于检验这些理论。
-
@r2evans 我想发布数据,但每年都在 3.7Gb 或更多
-
使用假数据。它们不必很大,但至少生成 2-3 个
data.frames,每个有 1-2 行和某种形式的差异,其中所有名称都在一个而不是在另一个。