【问题标题】:How to get Min, Max and Length between dates for each year?如何获取每年日期之间的最小值、最大值和长度?
【发布时间】:2019-11-02 06:27:23
【问题描述】:

我有一个 RDD[String] 类型的 rdd 作为示例,这里是它的一部分:

1990,1990-07-08
1994,1994-06-18
1994,1994-06-18
1994,1994-06-22
1994,1994-06-22
1994,1994-06-26
1994,1994-06-26
1954,1954-06-20
2002,2002-06-26
1954,1954-06-23
2002,2002-06-29
1954,1954-06-16
2002,2002-06-30
...

结果: (1982,52) (2006,64) (1962,32) (1966,32) (1986,52) (2002,64) (1994,52) (1974,38) (1990,52) (2010,64) (1978,38) (1954,26) (2014,64) (1958,35) (1998,64) (1970,32)

I group it nicely, but my problem is this v.size part, I do not know to to calculate that length.

Just to put it in perspective, here are expected results:

It is not a mistake that there is two times for 2002. But ignore that.

【问题讨论】:

    标签: scala date max rdd min


    【解决方案1】:

    定义日期格式:

    val formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd")
    

    和顺序:

    implicit val localDateOrdering: Ordering[LocalDate] = Ordering.by(_.toEpochDay)
    

    创建一个接收“v”并返回 MAX(date_of_matching_year) - MIN(date_of_matching_year)) = LENGTH(以天为单位)的函数:

    def f(v: Iterable[Array[String]]): Int = {
        val parsedDates = v.map(LocalDate.parse(_(1), formatter))
        parsedDates.max.getDayOfYear - parsedDates.min.getDayOfYear
    

    然后将 v.size 替换为 f(v)

    【讨论】:

    • v的类型是什么?五:..?我试过int,但map不适用于int
    • scala scala> val evega = concat.map(_.split(",")).keyBy(_(0)).groupByKey().map{case (k, v) => (k, f(v))} <console>:36: error: type mismatch; found : Iterable[Array[String]] required: Array[String] val evega = concat.map(_.split(",")).keyBy(_(0)).groupByKey().map{case (k, v) => (k, f(v))} ^ 我试过了
    • 而函数是:scala scala> def f(v: Array[String]): Int = { | val parsedDates = v.map(LocalDate.parse(_, formatter)) | parsedDates.max.getDayOfYear - parsedDates.min.getDayOfYear} f: (v: Array[String])Int
    • 我已经编辑了我的答案,V 是与相同键匹配的所有行的集合。由于每一行都是一个数组(因为 .split(","))它是一个 Iterable[Array[String]] 。现在我在回答中所做的更改是遍历所有行(使用平面图)并解析第二列上的日期并返回它。
    • 为什么会这样? scala <console>:37: error: missing parameter type for expanded function ((x$1) => x$1(1)) val parsedDates = v.flatMap(LocalDate.parse(_(1), formatter))
    猜你喜欢
    • 2014-12-19
    • 1970-01-01
    • 2019-05-31
    • 2020-04-30
    • 1970-01-01
    • 2016-07-23
    • 2014-10-17
    • 2021-12-15
    • 2021-05-30
    相关资源
    最近更新 更多