未能在项目 spark-core_2.11 上执行目标答案

【问题标题】：Failed to execute goal on project spark-core_2.11未能在项目 spark-core_2.11 上执行目标
【发布时间】：2017-12-05 08:53:59
【问题描述】：

, Spark 代码编译操作在少数机器上失败，而相同的源代码在少数其他机器上通过。 请检查 Centos (4.10.12-1.el7.elrepo.x86_64) 上的错误 ./build/mvn -X -DskipTests -Dscala.lib.directory=/usr/share/scala -pl 核心编译

INFO] Building Spark Project Core 2.2.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.804 s
[INFO] Finished at: 2017-12-04T23:18:58-08:00
[INFO] Final Memory: 43M/1963M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project spark-core_2.11: Could not resolve dependencies for project org.apache.spark:spark-core_2.11:jar:2.2.2-SNAPSHOT: The following artifacts could not be resolved: org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT, org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT: Failure to find org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT in http://artifact.eng.stellus.in:8081/artifactory/libs-snapshot was cached in the local repository, resolution will not be reattempted until the update interval of snapshots has elapsed or updates are forced -> [Help 1]

注意：相同的源代码在另一台 CentOS 机器上传递（3.10.0-514.el7.x86_64） ./build/mvn -DskipTests -Dscala.lib.directory=/usr/share/scala -pl 核心编译

[INFO] — maven-compiler-plugin:3.7.0:compile (default-compile) @ spark-core_2.11 —
[INFO] Not compiling main sources
[INFO]
[INFO] — scala-maven-plugin:3.2.2:compile (scala-compile-first) @ spark-core_2.11 —
[INFO] Using zinc server for incremental compilation
[info] Compile success at Dec 4, 2017 11:17:34 PM [0.331s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.663 s
[INFO] Finished at: 2017-12-04T23:17:34-08:00
[INFO] Final Memory: 52M/1297M
[INFO] ------------------------------------------------------------------------

【问题讨论】：

你从哪里得到2.2.2-SNAPSHOT？我不记得以前见过它。

标签： java maven apache-spark jar

【解决方案1】：

如果构建仅在特定机器上有效，则意味着您已经缺少对 .m2 本地存储库的依赖。如果您将删除 .m2 下的所有目录（在 centos 机器上），则构建将失败。

您应该添加所有缺少的依赖项。

[WARNING] The POM for org.apache.spark:spark-launcher_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-network-common_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-network-shuffle_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-unsafe_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:2.2.2-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.spark:spark-tags_2.11:jar:tests:2.2.2-SNAPSHOT is missing, no dependency information available

【讨论】：

感谢您的回复。在我们的例子中，存储库不是空的，但构建仍然无法正常工作。因此，我将 .m2/repository 从构建已通过 clean 的机器复制到失败的服务器。执行编译操作后，复制 .m2/repository 后构建成功运行。现在我担心的是，如果我没有任何机器之前构建已经通过 clean 并且它作为第一次尝试在新安装的服务器上执行编译操作并且我没有任何来自其他服务器的 .m2/repo 备份我该如何解决这个问题。

【解决方案2】：

以下补丁修复编译错误

diff --git a/pom.xml b/pom.xml
index cc48ee794e..ef0b01bec8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2518,6 +2518,13 @@
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-jar-plugin</artifactId>
           <version>3.1.2</version>
+                 <executions>
+                   <execution>
+                     <goals>
+                       <goal>test-jar</goal>
+                     </goals>
+                   </execution>
+                 </executions>
         </plugin>
         <plugin>
           <groupId>org.apache.maven.plugins</groupId>

请参阅此处 stackoverflow.com/questions/4786881 why is "test-jar" dependency required for "mvn compile"

另一种可能：您有一个执行 <maven.test.skip>true</maven.test.skip> 的 Maven 配置文件（它会跳过测试编译和执行）
代替 <skipTests>true</skipTests> （跳过执行，但编译测试）

【讨论】：