SQLite3 如何计算差异变化

Question

我有一个中等大小的数据库（当时 400,000 行），其中包含一个具有以下架构的测量表：

CREATE TABLE `Measurements` (
`timestamp` timestamp,
`timetick`  INTEGER,
`Sensor1`   REAL,
`Sensor2`   REAL,
PRIMARY KEY(timestamp));

随着时间戳增加（时间戳增加不是恒定的，存在间隙和延迟，但时间戳保证是单调的），通常时间戳也会增加，但在某些情况下它会重置为一个小但不可预测的值。我需要找到所有这样的行。我使用了以下查询（灵感来自Finding the difference in rows in query using SQLite）：

select r0,r1,a,b,rd,d from 
(select M0.rowid as r0,
  M1.rowid as r1,
  M0.timestamp as a,
  M1.timestamp as b,
  min(M1.timestamp)-M0.timestamp as rd,
  M1.timetick-M0.timetick as d
  from Measurements M0,Measurements M1
  where M1.timestamp>M0.timestamp group by M0.timestamp
) where d<0;

这可行，但需要几个小时，而 python 中的相同工作在 30 秒内完成。然而，这是一项非常常见的任务，科学家一直在计算衍生品，而金融专业人士则在计算价格差异。应该有一种有效的方法来做到这一点。
感谢您的帮助和 cmets。

Answer 1

带有 GROUP BY 的连接很难优化。

最好使用相关子查询来查找相应的下一行：

SELECT m0.rowid AS r0,
       m1.rowid AS rn,
       m0.timestamp AS a,
       m1.timestamp AS b,
       m1.timestamp - m0.timestamp AS rd,
       m1.timetick - m0.timetick AS d
FROM (SELECT rowid,     -- This is the core query attaching to each row
             timestamp, -- the rowid of its next
             timetick,
             (SELECT rowid
              FROM measurements
              WHERE timestamp > m.timestamp
              ORDER BY timestamp
              LIMIT 1
             ) AS r1
      FROM Measurements AS m
     ) AS m0
JOIN measurements AS m1 ON m0.r1 = m1.rowid
WHERE m1.timetick - m0.timetick < 0;

如果时间戳是整数，则将该列设为 INTEGER PRIMARY KEY 以避免额外的索引查找。