【问题标题】:Building sparkr with -Psparkr error使用 -Psparkr 错误构建 sparkr
【发布时间】:2015-09-24 13:22:36
【问题描述】:

我正在 Windows 系统上使用 cygwin 构建 spark。当我这样做时,我可以很好地构建并且它在之后工作

git clone https://github.com/apache/spark
dos2unix spark/build/mvn
build/mvn -DskipTests clean package

但是当我尝试使用 -Psparkr 配置文件时

build/mvn -DskipTests -Psparkr clean package

我收到一个错误,我尝试使用 -X 和 -e 选项,但似乎没有任何迹象表明它失败了。

[DEBUG] Executing command line: [cmd, /c, ..\R\install-dev.bat]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [  5.613 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 27.617 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 16.601 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 12.118 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 22.492 s]
[INFO] Spark Project Core ................................. FAILURE [03:19 min]
[INFO] Spark Project Bagel ................................ SKIPPED
[INFO] Spark Project GraphX ............................... SKIPPED
[INFO] Spark Project Streaming ............................ SKIPPED
[INFO] Spark Project Catalyst ............................. SKIPPED
[INFO] Spark Project SQL .................................. SKIPPED
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Twitter ..................... SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Project External MQTT ........................ SKIPPED
[INFO] Spark Project External MQTT Assembly ............... SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:44 min
[INFO] Finished at: 2015-09-24T09:37:59-04:00
[INFO] Final Memory: 46M/490M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.10: Command execution failed. Process exited with an error: -1 (Exit value: -1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.10: Command execution failed.
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed.
        at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:276)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        ... 20 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: -1 (Exit value: -1)
        at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
        at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
        at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:660)
        at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:265)
        ... 22 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :spark-core_2.10

关于它为什么会失败的任何线索?

【问题讨论】:

  • 使用 -X 参数运行 Maven 并发布堆栈跟踪。
  • 已将其添加到原始问题中,谢谢
  • 似乎cmd 找不到文件..\R\install-dev.bat。它存在吗?
  • 是的,我刚刚看到 Rtools 不在我的路径中,所以添加它并重试

标签: maven apache-spark cygwin sparkr


【解决方案1】:

可能是因为缺少 -Psparkr 编译选项所需的对 R 语言的依赖。

在 Debian GNU/Linux(尤其是 8/jessie)上,您需要运行 apt-get install r-base r-base-dev aptitude install r-base r-base-dev

如果您已安装aptitude 版本,则首选。

【讨论】:

    【解决方案2】:

    https://github.com/apache/spark/tree/master/R 阅读 SparkR 的 README.md。

    cd 源代码目录 (spark-1.6.1/R) 并运行 ./install-dev.sh

    然后,运行build/mvn -DskipTests -Psparkr package 就可以了

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2017-12-10
      • 2016-01-04
      • 2016-02-17
      • 1970-01-01
      • 2012-09-19
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多