逻辑上将一个流分成不相交的分区,每个分区包含相同键的元素。在内部,这是通过散列分区来实现的

object Keyby {
  def main(args: Array[String]): Unit = {
    val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
    env.setParallelism(3)
    val textStream: DataStream[String] = env.socketTextStream("localhost" , 12345)
    val flatMap_data: DataStream[String] = textStream.flatMap(line => line.split("\t"))
    val map_data: DataStream[(String, Int)] = flatMap_data.map(line => (line , 1))
    //TODO 逻辑上将一个流分成不相交的分区,每个分区包含相同键的元素。在内部,这是通过散列分区来实现的
    val keyByData: KeyedStream[(String, Int), String] = map_data.keyBy(line => line._1)
    keyByData.writeAsText("keyByData")
    env.execute()
  }
}

 

相关文章:

  • 2022-12-23
  • 2021-05-25
  • 2021-11-18
  • 2021-05-27
  • 2022-12-23
  • 2022-12-23
  • 2022-12-23
猜你喜欢
  • 2022-12-23
  • 2021-04-09
  • 2021-07-14
  • 2021-06-01
  • 2022-12-23
  • 2022-01-07
  • 2022-12-23
相关资源
相似解决方案