【发布时间】:2020-01-18 15:01:53
【问题描述】:
我正在尝试在 except/else 中使用累加器(变量):
更新代码
classifier = UtteranceClassifier()
sc = SparkContext("local[2]")
offensives = sc.accumulator(0)
total = sc.accumulator(0)
# Create Spark session
def get_spark_session():
return SparkSession \
.builder \
.master(master) \
.appName(appName) \
.getOrCreate()
def classify(utterance):
global offensives
global total
total += 1
print("total: ", total)
try:
return classifier.classify(utterance)
except Exception:
offensives += 1
print("offensives: ", offensives)
def main():
spark: SparkSession = get_spark_session()
....
# Clean text
df_clean = df.select((f.lower(f.regexp_replace('utterance', "[^a-zA-Z\\s]", "")).alias('utterance_text')))
# Tokenize text
tokenizer = Tokenizer(inputCol='utterance_text', outputCol='utterance_token')
df_words_token = tokenizer.transform(df_clean).select('utterance_token')
# Remove stop words
remover = StopWordsRemover(inputCol='utterance_token', outputCol='utterance_clean')
df_no_stopwords = remover.transform(df_words_token).select('utterance_clean')
#df_no_stopwords.show(truncate=False)
classify_udf = f.udf(classify, StringType())
# Classify utterance
df_no_stopwords = df_no_stopwords.withColumn("offensive", classify_udf(f.col('utterance_clean')))
df_no_stopwords.show(truncate=False)
print("Offensives: ", offensives.value)
print("Total: ", total.value)
我对变量总数和进攻性都有这样的编译错误:
检查信息:此检查检测到应该解析但不...
【问题讨论】:
-
请贴出实际代码而不仅仅是图片
-
无图像:将您的代码和相关错误发布为文本。
-
看来您正试图在
python中进行c编程。没有像num_of_offensive++或total++这样的语法。存在于python中。 -
如果我使用相同:num_of_offensive=num_of_offensive+1 !!
-
变量范围有问题。
标签: python python-3.x pyspark