时间戳之间的子集记录

Question

我有两个数据框，trips 是具有唯一 id 的自行车所进行的独特行程，intervals 则每 10 分钟显示一次自行车 ID 的位置。如果time 介于start 和finish 和bike_ids 之间，我的目标是删除 记录intervals。时间为posixCT类，原始数据帧有数十万条记录

例如，下面这两个数据集的结果应该是：

> trips
  bike_id               start              finish
1       1 2017-11-22 15:52:36 2017-11-22 17:47:53
2       2 2017-11-22 16:05:44 2017-11-22 16:23:25
3       3 2017-11-22 16:31:06 2017-11-22 17:11:20


  > intervals
                      time bike_id
    3  2017-11-22 16:00:03       1
    4  2017-11-22 16:10:03       1
    5  2017-11-22 16:20:02       1
    6  2017-11-22 16:30:02       1
    7  2017-11-22 16:40:03       1
    8  2017-11-22 16:50:02       1
    9  2017-11-22 17:00:02       1
    10 2017-11-22 17:10:02       1
    11 2017-11-22 17:20:03       1
    12 2017-11-22 17:30:03       1
    13 2017-11-22 16:00:03       2
    14 2017-11-22 16:10:03       2
    15 2017-11-22 16:20:02       2
    16 2017-11-22 16:30:02       2
    17 2017-11-22 16:40:03       2
    18 2017-11-22 16:50:02       2
    19 2017-11-22 17:00:02       2
    20 2017-11-22 17:10:02       2
    21 2017-11-22 17:20:03       2
    22 2017-11-22 17:30:03       2
    23 2017-11-22 16:30:02       3
    24 2017-11-22 16:40:03       3
    25 2017-11-22 16:50:02       3
    26 2017-11-22 17:00:02       3
    27 2017-11-22 17:10:02       3
    28 2017-11-22 17:20:03       3
    29 2017-11-22 17:30:03       3

结果

  > outcome
                      time bike_id
    13 2017-11-22 16:00:03       2
    16 2017-11-22 16:30:02       2
    17 2017-11-22 16:40:03       2
    18 2017-11-22 16:50:02       2
    19 2017-11-22 17:00:02       2
    20 2017-11-22 17:10:02       2
    21 2017-11-22 17:20:03       2
    22 2017-11-22 17:30:03       2
    23 2017-11-22 16:30:02       3
    28 2017-11-22 17:20:03       3
    29 2017-11-22 17:30:03       3

不确定从哪里开始。任何关于从dplyr 或apply 函数开始的建议将不胜感激！

这里是示例数据：

> dput(intervals)
structure(list(time = structure(c(1511384403.94561, 1511385003.17654, 
1511385602.47887, 1511386202.99895, 1511386803.18361, 1511387402.98233, 
1511388002.69461, 1511388602.5818, 1511389203.52712, 1511389803.652, 
1511384403.94561, 1511385003.17654, 1511385602.47887, 1511386202.99895, 
1511386803.18361, 1511387402.98233, 1511388002.69461, 1511388602.5818, 
1511389203.52712, 1511389803.652, 1511386202.99895, 1511386803.18361, 
1511387402.98233, 1511388002.69461, 1511388602.5818, 1511389203.52712, 
1511389803.652), class = c("POSIXct", "POSIXt"), tzone = ""), 
    bike_id = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 
    2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3)), .Names = c("time", 
"bike_id"), row.names = 3:29, class = "data.frame")

> dput(trips)
structure(list(bike_id = c(1, 2, 3), start = structure(c(1511383956, 
1511384744, 1511386266), class = c("POSIXct", "POSIXt"), tzone = ""), 
    finish = structure(c(1511390873, 1511385805, 1511388680), class = c("POSIXct", 
    "POSIXt"), tzone = "")), .Names = c("bike_id", "start", "finish"
), row.names = c(NA, 3L), class = "data.frame")

Answer 1

我是data.table包的新手，所以请仔细测试以下方法。

我选择data.table 而不是dplyr 的原因是因为此任务需要按范围连接，而dplyr 目前无法执行。这是使用foverlaps 函数的解决方案。

library(data.table)

# Convert the data frame to data.table
setDT(intervals)
setDT(trips)

# Create a second column time2, which is the same as time
intervals[, time2 := time]

# Set keys in trips
setkey(trips, bike_id, start, finish)

# Conduct join by bike_id and time
# The columns in intervals used for join by time 
# needs to be in the last two in by.x
intervals2 <- foverlaps(intervals, trips, 
                        by.x = c("bike_id", "time", "time2"))

# Filter the ones with NA in start, which means no match
# And then selct the time and bike_id column
outcome <- intervals2[is.na(start)][, .(time, bike_id)] 
outcome

#                   time bike_id
#  1: 2017-11-22 16:00:03       2
#  2: 2017-11-22 16:30:02       2
#  3: 2017-11-22 16:40:03       2
#  4: 2017-11-22 16:50:02       2
#  5: 2017-11-22 17:00:02       2
#  6: 2017-11-22 17:10:02       2
#  7: 2017-11-22 17:20:03       2
#  8: 2017-11-22 17:30:03       2
#  9: 2017-11-22 16:30:02       3
# 10: 2017-11-22 17:20:03       3
# 11: 2017-11-22 17:30:03       3 

Answer 2

这可以通过一种非等值反连接来解决。

非 equi 连接自 1.9.8 版（CRAN 2016 年 11 月 25 日）起在 data.table 中可用，并且在许多情况下可用作 foverlaps() 的便捷替代品。特别是，foverlaps() 要求对第二个参数进行键控，而 non-equi join 同样适用于未键控和键控的 data.tables。

首先，非等值连接用于识别在trips 的start 和finish 倍内的intervals 行的索引。然后，这些行从intervals中删除

library(data.table)
tmp <- setDT(intervals)[setDT(trips), on = .(bike_id, time >= start, time <= finish), 
                        which = TRUE]
intervals[!tmp]

                   time bike_id
 1: 2017-11-22 16:00:03       2
 2: 2017-11-22 16:30:02       2
 3: 2017-11-22 16:40:03       2
 4: 2017-11-22 16:50:02       2
 5: 2017-11-22 17:00:02       2
 6: 2017-11-22 17:10:02       2
 7: 2017-11-22 17:20:03       2
 8: 2017-11-22 17:30:03       2
 9: 2017-11-22 16:30:02       3
10: 2017-11-22 17:20:03       3
11: 2017-11-22 17:30:03       3

tmp 包含要删除的行的索引：

tmp

 [1]  1  2  3  4  5  6  7  8  9 10 12 13 22 23 24 25

Answer 3

这是我的答案。 trips 是参考数据集。

matched() 是一个函数，用于匹配从 trips 到 intervals 的开始和结束。

回答

trips <- data.frame(bike_id = 1:3, 
                    start = as.POSIXct(c("2017-11-22 15:52:36", "2017-11-22 16:05:44", "2017-11-22 16:31:06")),
                    finish = as.POSIXct(c("2017-11-22 17:47:53","2017-11-22 16:23:25","2017-11-22 17:11:20")))%>%
         mutate(start = as.numeric(start),
                finish = as.numeric(finish))


matched <- function(var1, var2, df1, df2){
return(df2[,var1][match(df1[,var2],df2[,var2])])
}



intervals%>%
mutate(time_num = as.numeric(time),
       start = matched("start", "bike_id", intervals , trips),
       finish = matched("finish", "bike_id", intervals , trips))%>%
filter(time_num < start | time_num > finish)%>%
select(time, bike_id)


                  time bike_id
1  2017-11-22 16:00:03       2
2  2017-11-22 16:30:02       2
3  2017-11-22 16:40:03       2
4  2017-11-22 16:50:02       2
5  2017-11-22 17:00:02       2
6  2017-11-22 17:10:02       2
7  2017-11-22 17:20:03       2
8  2017-11-22 17:30:03       2
9  2017-11-22 16:30:02       3
10 2017-11-22 17:20:03       3
11 2017-11-22 17:30:03       3

由于某种奇怪的原因，我无法让 between() 工作。稍后我会看到。