Clickhouse: topK by uniqs or sum of other column

Question

我们将会话存储在 Clickhouse 中。一行（除其他外）有一个城市、一个持续时间、一个 IP 和一个代理列。在一个聚合中，我们按页面分组，并按 IP 和代理计算持续时间和 uniq 的总和。我们还汇总了前 5 个城市。但是在选择前 5 个城市之前，城市是按数据库中出现的次数排序的。是否可以使用 uniq 访问者（如代理/IP 组合所示）或持续时间总和来确定城市的顺序？

编辑（添加特定查询和更多解释）：

          SELECT page, day,
            CAST(uniqExact(ip, agent) AS UInt16) AS uniqs,
            topKIf(5)(city, city <> '') AS top_cities,
            sum(duration) AS total_duration
          FROM pageviews
          WHERE day = toDate('2019-12-24')
          GROUP BY page

那么top_cities 是由给定城市的浏览量决定的。我希望 top_cities 由每个城市的 sum(duration) 或每个城市的 uniq ip/agent 组合数确定。

我知道我可以GROUP BY page, city, ip, agent 并在一个额外的步骤中进行最终聚合，但这只是数据集需要很长时间。

Answer 1

topK 返回一个由大多数 频繁 值组成的数组，因此它在这里无济于事。

看起来需要使用这样的直接方式：

SELECT
    page,
    groupArray((city, metric)) AS cityMetricArray,

    /* Assign each City the numeric unique ID. 
       If your dataset contains CityId then use it instead of this artificial key. */
    arrayMap((x, id) -> (x.1, x.2, id), cityMetricArray, arrayEnumerateDense(arrayMap(x -> (x.1), cityMetricArray))) AS cityMetricCityIdArray,

    /* Calculate the sum of metrics for each city. 
       Unfortunately sumMap-function accepted only numeric array as key-array, otherwise, passing an array with city names as keys would make code more simple.  */
    arrayReduce('sumMap', [arrayMap(x -> x.3, cityMetricCityIdArray)], [arrayMap(x -> x.2, cityMetricCityIdArray)]) AS cityMetricSumArray,

    /* Take 5-top cities Ids. */
    arrayReverseSort((cityId, sumMetric) -> sumMetric, cityMetricSumArray.1, cityMetricSumArray.2) AS cityIds,
    arraySlice(cityIds, 1, 5) AS topNCityIds,

    /* Map cityIds to city names. */
    arrayMap(cityId -> arrayFirst(x -> x.3 = cityId, cityMetricCityIdArray).1, topNCityIds) AS topCities
FROM
(   /* test data */
    SELECT
        data.1 AS city,
        data.2 AS metric,
        'page' AS page
    FROM
    (
        SELECT arrayJoin([
          ('city1', 11), ('city2', 11), ('city3', 11), 
          ('city4', 11), ('city2', 11), ('city4', 22), 
          ('city5', 5), ('city6', 22), ('city7', 10)]) AS data
    )
)
GROUP BY page
FORMAT Vertical

/* Result:
page:                  page
cityMetricArray:       [('city1',11),('city2',11),('city3',11),('city4',11),('city2',11),('city4',22),('city5',5),('city6',22),('city7',10)]
cityMetricCityIdArray: [('city1',11,1),('city2',11,2),('city3',11,3),('city4',11,4),('city2',11,2),('city4',22,4),('city5',5,5),('city6',22,6),('city7',10,7)]
cityMetricSumArray:    ([1,2,3,4,5,6,7],[11,22,11,33,5,22,10])
cityIds:               [4,2,6,1,3,7,5]
topNCityIds:           [4,2,6,1,3]
topCities:             ['city4','city2','city6','city1','city3']
*/