编译 apache sparkSql 时出错答案

【问题标题】：Error while compiling apache sparkSql编译 apache sparkSql 时出错
【发布时间】：2014-10-12 02:44:56
【问题描述】：

我正在尝试执行下面的 sparkSQL 示例，

https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQL.java

我的 pom.xml（我也添加了 sql 依赖项，我卡住了，请帮忙）

<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>edu.berkeley</groupId>
  <artifactId>simple-project</artifactId>
  <packaging>jar</packaging>
  <version>1.0</version>
  <name>Simple Project</name>
  <repositories>
    <repository>
      <id>Akka repository</id>
      <url>http://repo.akka.io/releases</url>
    </repository>
  </repositories>
  <dependencies>
    <dependency> <!-- Spark dependency -->
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.10</artifactId>
      <version>1.0.2</version>
    </dependency>
    <dependency> <!-- Spark dependency -->
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-sql_2.10</artifactId>
      <version>1.0.2</version>
    </dependency>
  </dependencies>
</project>

我刚刚将文件重命名为 SimpleApp.java

编译时出现以下错误..请指教

[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Simple Project 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ simple-project ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Volumes/Official/spark-1.0.2-bin-hadoop2/try/simple-project/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ simple-project ---
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to /Volumes/Official/spark-1.0.2-bin-hadoop2/try/simple-project/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /Volumes/Official/spark-1.0.2-bin-hadoop2/try/simple-project/src/main/java/SimpleApp.java:[61,16] error: cannot find symbol
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.801 s
[INFO] Finished at: 2014-08-19T10:30:31+05:30
[INFO] Final Memory: 21M/232M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project simple-project: Compilation failure
[ERROR] /Volumes/Official/spark-1.0.2-bin-hadoop2/try/simple-project/src/main/java/SimpleApp.java:[61,16] error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

【问题讨论】：

似乎程序在 schemaPeople.registerTempTable("people"); 处抛出错误；
我一直在检查 Spark JavaDoc.. 无法获得任何线索.. 我在这里遗漏了什么

标签： java apache-spark

【解决方案1】：

我发现了问题：

在示例中

https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQL.java

替换schemaPeople.registerTempTable("people");这一行

schemaPeople.registerAsTable("people");

使用mvn compile 和mvn package 编译和打包并运行

bin/spark-submit --class "org.apache.spark.examples.sql.SimpleApp" --master local[4] try/simple-project/target/simple-project-1.0.jar

【讨论】：