maven依赖java.util.function没有正确加载函数类

Question

当我在 maven 中构建它时，它编译成功。

package com.hf.arm;

import org.apache.spark.SparkConf;
import java.util.Arrays;
import java.util.List;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.mllib.fpm.AssociationRules;
import org.apache.spark.mllib.fpm.FPGrowth;
import org.apache.spark.mllib.fpm.FPGrowthModel;
public class App 
{
    public static void main( String[] args )
    {
        SparkConf conf = new SparkConf().setAppName("FP-growth Example");
        JavaSparkContext sc = new JavaSparkContext(conf);
        JavaRDD<String> data = sc.textFile("/Users/lincolnsmith/Table_csv/sample_fpgrowth.txt");
    }
}

我需要添加一些代码，所以新脚本如下所示：

package com.hf.arm;

import org.apache.spark.SparkConf;
import java.util.Arrays;
import java.util.List;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.mllib.fpm.AssociationRules;
import org.apache.spark.mllib.fpm.FPGrowth;
import org.apache.spark.mllib.fpm.FPGrowthModel;

public class App 
{
    public static void main( String[] args )
    {
    SparkConf conf = new SparkConf().setAppName("FP-growth Example");
    JavaSparkContext sc = new JavaSparkContext(conf);
    JavaRDD<String> data = sc.textFile("/Users/lincolnsmith/Table_csv/sample_fpgrowth.txt");
    JavaRDD<List<String>> transactions = data.map(
      new Function<String, List<String>>() {
        public List<String> call(String line) {
          String[] parts = line.split(" ");
          return Arrays.asList(parts);
        }
      }
    );
    FPGrowth fpg = new FPGrowth()
      .setMinSupport(0.2)
      .setNumPartitions(10);
    FPGrowthModel<String> model = fpg.run(transactions);
    for (FPGrowth.FreqItemset<String> itemset: model.freqItemsets().toJavaRDD().collect()) {
      System.out.println("[" + itemset.javaItems() + "], " + itemset.freq());
    }
    double minConfidence = 0.8;
    for (AssociationRules.Rule<String> rule
        : model.generateAssociationRules(minConfidence).toJavaRDD().collect()) {
      System.out.println(
        rule.javaAntecedent() + " => " + rule.javaConsequent() + ", " + rule.confidence());
    }

    }
}

但我收到此错误： “找不到符号符号：类函数” 显然我缺少包 java.utils.function，所以我导入了这个

import java.util.function.Function;

然后我将依赖项添加到我的 pom.xml：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.hf.arm</groupId>
  <artifactId>arm</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>arm</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency> <!-- Spark dependency -->
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.10</artifactId>
        <version>1.5.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.10</artifactId>
    <version>1.5.1</version>
    </dependency>
    <dependency>
    <groupId>net.sf.m-m-m</groupId>
    <artifactId>mmm-util-backport-java.util.function</artifactId>
    <version>1.0.1</version>
    </dependency>
  </dependencies>
<build>
<plugins>
  <plugin>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.3</version>
      <configuration>
          <source>1.7</source>
          <target>1.7</target>
      </configuration>
  </plugin>
</plugins>
</build>
</project>

但现在我继续得到错误输出：

Lincolns-MacBook-Pro:arm lincolnsmith$ mvn package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building arm 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ arm ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/lincolnsmith/Table_csv/arm/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ arm ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to /Users/lincolnsmith/Table_csv/arm/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,11] '.' expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,12] ';' expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,16] class, interface, or enum expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,17] class, interface, or enum expected
[INFO] 4 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.241 s
[INFO] Finished at: 2015-10-14T10:05:13-04:00
[INFO] Final Memory: 19M/437M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project arm: Compilation failure: Compilation failure:
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,11] '.' expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,12] ';' expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,16] class, interface, or enum expected
[ERROR] /Users/lincolnsmith/Table_csv/arm/src/main/java/com/hf/arm/App.java:[6,17] class, interface, or enum expected
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

我找不到此依赖项的其他 maven 源，我已尝试清理并清空 my.m2 目录。 java.utils.function 是否有不同的依赖源可以用于 maven 还是有其他问题？

这是我的 java 版本：

Lincolns-MacBook-Pro:arm lincolnsmith$ java -version
java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)

Answer 1

实际上你不需要 backport 包，因为 Function 属于 Spark API，而不是 Java 8 API。试试这个：

JavaRDD<List<String>> transactions = data.map(
    new org.apache.spark.api.java.function.Function<String, List<String>>() {
        public List<String> call(String line) throws Exception {
            String[] parts = line.split(" ");
            return Arrays.asList(parts);
        }
    }
);

然后您可以使用正确的类路径构建和运行您的应用程序：

mvn clean compile exec:java -Dexec.mainClass="com.hf.App"

而且效果很好（在我的情况下达到预期效果）：

...
org.apache.spark.SparkException: A master URL must be set in your configuration
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:394)
        ...