组合查询结果的最有效方法

Question

这个问题有点探索性，因为我决定在尝试实施之前征求意见/建议，以期更及时地获得结果。我有一个数据库，以 1 分钟的格式存储商品和股票的日内数据。这背后的基本原理是，如果我有 1 分钟柱，我可以创建任何我想要的时间序列柱（即 5 分钟、15 分钟、60 分钟等）

我考虑过这样的路线，例如，在 15 分钟图表上，我会简单地遍历给定符号的整个结果子集，查找时间戳为 :01,:16,:31:46 的行开始捕获打开、关闭、maxhigh 和 minlow 以及累积量，并从该合并中创建新数据点。问题在于，对于大宗商品，尤其是 24 小时交易的商品，并非总是每分钟都有一根柱线，因此可能会出现给定交易品种没有 :16 柱线或 :31 柱线的情况。这可能会破坏正确获取条形数据的整个顺序。这也消除了一次仅抓取 15 根柱线并捕获开盘、收盘、最高和最低以及累积交易量的可能性。 （当然，表格包含symbol、datetime、open、high、low、close、volume）

为了使上述任何一项工作，我必须每晚“修复”数据库，方法是检查缺失的条形并复制先前的条形以为缺失的行创建一个条形。这不是首选方法，但如有必要，我会实现。

我正在寻找有关正确路径的任何指导，无论是从过去的经验还是从此处阅读有关此问题的信息。

更新： 这是我想出的代码，但它有点慢，从数据库表中的 1 分钟数据创建 2 个月的 30 分钟条大约需要 30 秒。

Partial Class testintracharts
Inherits Page
<WebMethod>
Public Shared Function GetBars(ByVal symbol As String, ByVal seriesInterval As Integer) As List(Of ArrayList)

    Dim barsList As New List(Of ArrayList)
    'replace with TD model
    Using ctx As New BATLEntities()

        ctx.Configuration.AutoDetectChangesEnabled = False

        'get all days for a given symbol, ordered from oldest to newest
        Dim dateList As List(Of DateTime) = GetDistinctDates(ctx, symbol, seriesInterval).ToList()

        'cache data series toa list for the given symbol
        Dim seriesList As List(Of tsintrachart) = GetSymbolSeries(ctx, symbol)

        If Not dateList Is Nothing And dateList.Any() Then
            For Each seriesDate As DateTime In dateList
                Dim curTime As TimeSpan = New TimeSpan(0, 1, 0)
                Dim maxTime As TimeSpan = New TimeSpan(24, 0, 0)

                'loop through the data series for the given day
                'series start at 00:01:00
                'series ends at 00:00:00 (next day)

                While curTime < maxTime
                    Dim seriesMax As TimeSpan = curTime.Add(New TimeSpan(0, seriesInterval - 1, 0))

                    'get the data chunk based on series interval
                    'special condition when seriesmax reaches 24:00:00, in TimeSpan this becomes 1.00:00:00 (SQL doesn't like this)
                    'query needs to incorporate 00:00:00 of next day as the last entry for this time series

                    Dim data As List(Of tsintrachart) = Nothing
                    If TimeSpan.Compare(seriesMax, maxTime) = 0 Then
                        Dim nextSeriesDate As DateTime = seriesDate.AddDays(1)
                        data = GetDataSeries(seriesList, seriesDate, curTime, nextSeriesDate, New TimeSpan(0, 0, 0))
                    Else
                        data = GetDataSeries(seriesList, seriesDate, curTime, seriesDate, seriesMax)
                    End If

                    If Not data Is Nothing And data.Any() Then

                        Dim lastbarnum As Integer = data.Count - 1
                        Dim intradayDatum As New ArrayList()
                        With intradayDatum
                            .Add(DateTimeToUnixTimestamp(seriesDate.Add(seriesMax))) 'date
                            .Add(data(0).Open)
                            '.Add((From d In data Where TimeSpan.Compare(d.Time, curTime) = 0 Select d.Open).FirstOrDefault()) 'open
                            .Add((From d In data Select d.High).Max()) 'high
                            .Add((From d In data Select d.Low).Min()) 'low 
                            .Add(data(lastbarnum).Close)
                            '.Add((From d In data Where TimeSpan.Compare(d.Time, seriesMax) = 0 Select d.Close).FirstOrDefault()) 'close or last sale
                            .Add((From d In data Select d.Volume).Sum()) 'volume 
                        End With
                        barsList.Add(intradayDatum)

                    End If

                    'update current series start time, move to next series chunk
                    curTime = curTime.Add(New TimeSpan(0, seriesInterval, 0))
                End While

            Next
        End If
    End Using

    Return barsList
End Function

Private Shared Function GetDataSeries(ByRef list As List(Of tsintrachart), ByVal startDate As DateTime, ByVal startSpan As TimeSpan, ByVal endDate As DateTime, ByVal endSpan As TimeSpan) As List(Of tsintrachart)
    'LINQ, where symbol/time>=startRange/time<=endRange

    Dim series = From data In list
                 Where (TimeSpan.Compare(data.Time, startSpan) >= 0 AndAlso data.Date = startDate) _
                 And (TimeSpan.Compare(data.Time, endSpan) <= 0 AndAlso data.Date = endDate)
                Select data

    Return series.ToList()
End Function

Private Shared Function GetSymbolSeries(ByRef ctx As BATLEntities, ByVal symbol As String) As List(Of tsintrachart)
    Dim series = From data In ctx.tsintracharts
                 Where (data.Symbol = symbol)
                Select data

    Return series.ToList()
End Function

Private Shared Function GetDistinctDates(ByRef ctx As BATLEntities, ByVal symbol As String, ByVal interval As Integer) As IQueryable(Of DateTime)
    Dim numDates As Integer = TakeValue(interval)
    Dim dates = (From data In ctx.tsintracharts
                Where data.Symbol = symbol
                Select data.Date
                Order By [Date]).Distinct().Take(numDates)

    Return dates
End Function
Private Shared Function TakeValue(ByVal interval As Integer) As Integer
    Select Case interval
       Case 0 To 15
            Return 1440
        Case 16 To 30
            Return 720
        Case 31 To 60
            Return 528
        Case Else
            Return 400
    End Select
End Function
End Class

Answer 1

您可以创建一个包含所有可能的 1 分钟时间点的表。然后您可以OUTER JOIN 到该表以使服务器填补任何空白。表格是这样的：

CREATE TABLE TimePoints (
 DateTime DATETIME2(0) PRIMARY KEY
)

而且你必须填写几十年的数据。要查询，您将加入它：

SELECT *
FROM TimePoints
LEFT JOIN myOtherTable ON ...
WHERE TimePoints.DateTime >= (nowMinus15min) AND TimePoints.DateTime <= now