【问题标题】:Date diff in hive and the difference should be in hh:mm:ss配置单元中的日期差异,差异应该在 hh:mm:ss
【发布时间】:2017-08-23 16:25:43
【问题描述】:

我试图找出连续行中两个日期之间的差异。我在 hive 中使用窗口函数,即lag

但不同之处,即输出格式应为hh:mm:ss

例如:

  • 日期 1 是 2017-08-15 02:00:32
  • 日期 2 是2017-08-15 02:00:20

输出应该是:

00:00:12

我尝试的查询:

select from_unixtime(column_name),
(lag(unix_timestamp(from_unixtime(column_name)),1,0)
over(partition by column_name)-
unix_timestamp(from_unixtime(column_name))) as Duration from table_name;

但这会将输出返回为12(在上面的示例中)。

更新

我已将该列存储在具有 bigint 数据类型的表中。时间采用纪元格式。我们在查询中使用 from_unixtime 将其转换为可读日期。时间戳中的示例值

1502802618 1502786788

【问题讨论】:

    标签: date hadoop hive hiveql


    【解决方案1】:
    hive> with t as (select 1502802618 as ts1,1502786788 as ts2)
        > select  printf('%02d:%02d:%02d',(ts1 - ts2) div 3600,((ts1 - ts2) % 3600) div 60,((ts1 - ts2) % 3600) % 60) as diff
        > from    t
        > ;
    OK
    diff
    04:23:50
    

    【讨论】:

      【解决方案2】:

      只要时差小于 24 小时,答案就是相关的

      hive> with t as (select 1502802618 as ts1,1502786788 as ts2)
          > select  from_unixtime(to_unix_timestamp('0001-01-01 00:00:00')+(ts1 - ts2))  as diff
          > from    t
          > ;
      OK
      diff
      0001-01-01 04:23:50
      

      hive> with t as (select 1502802618 as ts1,1502786788 as ts2)
          > select  substr(from_unixtime(to_unix_timestamp('0001-01-01 00:00:00')+(ts1 - ts2)),12)   as diff
          > from    t
          > ;
      OK
      diff
      04:23:50
      

      【讨论】:

        【解决方案3】:

        只要时差小于 24 小时,答案就是相关的

        hive> with t as (select timestamp '2017-08-15 02:00:32' as ts1,timestamp '2017-08-15 02:00:20' as ts2)
            > select  ts1 - ts2   as diff
            > from    t
            > ;
        OK
        diff
        0 00:00:12.000000000
        

        给定时间戳

        hive> with t as (select timestamp '2017-08-15 02:00:32' as ts1,timestamp '2017-08-15 02:00:20' as ts2)
            > select  split(ts1 - ts2,'[ .]')[1]  as diff
            > from    t
            > ;
        OK
        diff
        00:00:12
        

        给定字符串

        hive> with t as (select '2017-08-15 02:00:32' as ts1,'2017-08-15 02:00:20' as ts2)
            > select  split(cast(ts1 as timestamp) - cast(ts2 as timestamp),'[ .]')[1]  as diff
            > from    t
            > ;
        OK
        diff
        00:00:12
        

        【讨论】:

        • 我尝试执行您的查询,但它抛出错误“没有与(时间戳,时间戳)类 org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPMinus 的匹配方法”。只是为了提供有关表架构的更多信息,我存储在 bigint 中的列,它位于纪元时间戳中。
        • 这是一个经过测试的代码。显然您的 Hive 版本不支持时间戳减法。
        • 我们正在使用 Cloudera-5.3.1/hive-0.13.1-cdh5.3.1。这个版本不支持?
        • @Shash,3 年前(2014 年 6 月 6 日)的版本?...否
        • @Shash, (1) 评论在正确的地方 (2) 你说的是基本的SQL,把ts1和@替换987654325@ 与相关表达式(列和滞后(...)超过(...))
        猜你喜欢
        • 1970-01-01
        • 2021-10-26
        • 2022-01-19
        • 2012-12-13
        • 1970-01-01
        • 1970-01-01
        • 1970-01-01
        • 1970-01-01
        • 1970-01-01
        相关资源
        最近更新 更多