【问题标题】:Apache Flink - WordCount - NoResult - PyFlinkApache Flink - WordCount - NoResult - PyFlink
【发布时间】:2022-01-24 01:52:50
【问题描述】:

我使用 PyFlink 开发了一个字数统计程序。该程序没有抛出任何错误,但没有提供所需的输出。根据代码,程序应该创建一个新的文本文件,但在执行时没有生成文件。请帮忙,我的代码附在下面。

from flink.plan.Constants import WriteMode
from flink.plan.Environment import get_environment
from flink.functions.FlatMapFunction import FlatMapFunction
from flink.functions.GroupReduceFunction import GroupReduceFunction
from pyflink import datastream

from pyflink.common import WatermarkStrategy, Encoder, Types
from pyflink.datastream import StreamExecutionEnvironment, RuntimeExecutionMode
from pyflink.datastream.connectors import (FileSource, StreamFormat, FileSink, OutputFileConfig, RollingPolicy)
                                       
class Tokenizer(FlatMapFunction):
    def flat_map(self, value, collector):
        super().__init__()
        for word in value.lower().split(","):
            if len(word)>1:
                collector.collect((word, 1))

if __name__ == '__main__':

env = get_environment()
env.set_parallelism(2)
data = env.read_text("h.txt")

tokenized = data.flat_map(Tokenizer())
count = tokenized.group_by(0).sum(1)
count.write_text("D:/Cyber Security/Apache Flink")

【问题讨论】:

    标签: apache-flink flink-streaming word-count pyflink


    【解决方案1】:

    尝试在程序末尾使用env.execute("Word Count Example...")。它开始你的执行。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2017-02-25
      • 2021-07-13
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2016-11-13
      • 2016-09-24
      • 1970-01-01
      相关资源
      最近更新 更多