【问题标题】:How to replace item in array with other values in spark?如何用火花中的其他值替换数组中的项目?
【发布时间】:2020-12-10 18:12:47
【问题描述】:
+--------------------------------+
|Subject |
+--------------------------------+
|[English, Math, Science, Spark] |
+--------------------------------+
|[English, History, Art] |
+--------------------------------+
我们如何在两行中用 ENGLISH 替换英语?
【问题讨论】:
标签:
arrays
apache-spark
replace
capitalization
【解决方案1】:
使用自定义 UDF 替换单词:
val replace = udf{ x: Seq[String] => x.map(y => if(y == "English") "ENGLISH" else y) }
val df2 = df.select(replace($"Subject").alias("Subject"))
df2.show(false)
+-------------------------------+
|Subject |
+-------------------------------+
|[ENGLISH, Math, Science, Spark]|
|[ENGLISH, History, Art] |
+-------------------------------+