Apache Hive 查询 HiveQL答案

【问题标题】：Apache Hive Query HiveQLApache Hive 查询 HiveQL
【发布时间】：2014-10-17 09:48:04
【问题描述】：

我正在学习 Hive，想编写一个优化的 HiveQL/SQL 查询

我的桌子是这样的：

CREATE TABLE sales (dealer VARCHAR(25), make VARCHAR(25), type VARCHAR(25), day INT);
INSERT INTO sales (dealer, make, type, day) VALUES
("Xyz", "Highlander", "SUV", "0"),
("Xyz", "Prius", "HATCH", "1"),
("Xyz", "Prius", "HATCH", "2"),
("Xyz", "Prius", "HATCH", "3"),
("Xyz", "Versa", "HATCH", "1"),
("Xyz", "Versa", "HATCH", "2"),
("Xyz", "Versa", "HATCH", "3"),
("Xyz", "S3", "SEDAN", "1"),
("Xyz", "S3", "SEDAN", "2"),
("Abc", "Forrester", "SUV", "1");

给定一个“经销商”D，我想计算过去每个“类型”的前 N 个“制造商”X天，在一个查询中。

SELECT dealer, make, type, COUNT(*) AS frequency FROM sales
WHERE day > 0 AND dealer LIKE 'Xyz' GROUP BY make, type
ORDER BY frequency DESC LIMIT 5

问题是当在“make”和“type”上使用 GROUP BY 时，我只会得到：

DEALER, MAKE, TYPE, COUNT
Xyz, Prius, Hatch, 3
Xyz, Versa, Hatch, 3
Xyz, S3, Sedan, 2
...

但我想要

Xyz, Prius, Hatch, 3
Xyz, S3, Sedan, 2
...

for EACH“输入”前 N 个。

有人可以帮我理解如何编写这样的查询吗？

SQL 小提琴 http://sqlfiddle.com/#!2/df9304/5

****更新****

似乎 rank() 会很有用：

Hive getting top n records in group by query

https://blogs.oracle.com/taylor22/entry/hive_0_11_may_15

HiveQL and rank()

【问题讨论】：

你运行了什么查询来得到你列出的结果？
我没有运行查询，但我对文档的理解：从表中选择经销商、品牌、类型、计数 (*) 作为频率 WHERE 日期 > 0 AND 经销商 == 'Xyz' GROUP BY 品牌, 输入 ORDER BY frequency DESC LIMIT 5
假设您想计算每种类型的前 5 个品牌...xyz, versa, hatch, 3 怎么不包含在该列表中？
@GoBrewers14，是的，你是对的，我举了一个前 1 的例子。有问题的更新。看rank()，看来会有用。

标签： sql hadoop hive hiveql top-n

【解决方案1】：

阅读更多文档和链接问题的提示后：

SELECT dealer, make, rank, type FROM (
    SELECT dealer, make, rank() OVER (PARTITION BY type ORDER BY count DESC) AS rank, type FROM (
        SELECT dealer, make, count(*) AS count, type FROM Sales WHERE dealer = "Xyz" GROUP BY dealer, type, make
    ) CountedSales
) RankedSales
WHERE RankedSales.rank < 3;

内部查询执行计数，中间查询执行 rank()，外部查询限制排名。

销售表内容

hive> select * from Sales;
OK
Xyz      Highlander      SUV    NULL
Xyz      Highlander      SUV    NULL
Xyz      Rouge   SUV    NULL
Xyz      Rouge   SUV    NULL
Xyz      Prius   HATCH  NULL
Xyz      Prius   HATCH  NULL
Xyz      Prius   HATCH  NULL
Xyz      Versa   HATCH  NULL
Xyz      S3      SEDAN  NULL
Xyz      S3      SEDAN  NULL
Xyz      S3      SEDAN  NULL
Xyz      A8      SEDAN  NULL
Xyz      A8      SEDAN  NULL
Xyz      A8      SEDAN  NULL
Xyz      A8      SEDAN  NULL
Time taken: 0.054 seconds, Fetched: 15 row(s)

现在是实际的查询。

hive> SELECT dealer, make, rank, type FROM (                                                                          
    >     SELECT dealer, make, rank() OVER (PARTITION BY type ORDER BY count DESC) AS rank, type FROM (
    >         SELECT dealer, make, count(*) AS count, type FROM Sales WHERE dealer = "Xyz" GROUP BY dealer, type, make
    >     ) CountedSales
    > ) RankedSales
    > WHERE RankedSales.rank < 3;
...
Execution completed successfully
MapredLocal task succeeded
OK
Xyz      Prius  1        HATCH
Xyz      Versa  2        HATCH
Xyz      A8     1        SEDAN
Xyz      S3     2        SEDAN
Xyz      Rouge  1        SUV
Xyz      Highlander     1        SUV
Time taken: 28.491 seconds, Fetched: 6 row(s)

【讨论】：