SBT：SPARK2.0.*：SCALA：未解决的依赖关系 org.glassfish.hk2 和 javax.validation答案

【问题标题】：SBT: SPARK2.0.*: SCALA: unresolved dependencies org.glassfish.hk2 and javax.validationSBT：SPARK2.0.*：SCALA：未解决的依赖关系 org.glassfish.hk2 和 javax.validation
【发布时间】：2017-04-18 09:05:58
【问题描述】：

尝试同时使用最新的Apache Spark 2.0.2 和Scala 2.11.7 时无法解决依赖关系

我需要使用最新版本的spark，因为它支持SparkSession。以下配置适用于spark 1.6.2。但我想要spark 2.0.*

这是我的依赖配置

scalaVersion := "2.11.7"

val sparkVersion = "2.0.2"

libraryDependencies ++= Seq(
  ws   ,
  "org.apache.spark" %% "spark-core" % sparkVersion,
  "org.apache.spark" %% "spark-sql" % sparkVersion,
  "org.apache.spark" %% "spark-streaming" % sparkVersion,
  "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0",
  specs2 % Test
)

这是错误信息：

Error:Error while importing SBT project:
[info] Resolving org.ow2.asm#asm-tree;4.1 ...
[info] Resolving org.ow2.asm#asm-analysis;4.1 ...
[info] Resolving org.ow2.asm#asm-util;4.1 ...
[info] Resolving jline#jline;2.12.1 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.glassfish.hk2#hk2-utils;2.22.2: not found
[warn]  :: org.glassfish.hk2#hk2-locator;2.22.2: not found
[warn]  :: javax.validation#validation-api;${javax.validation.version}: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] Total time: 81 s, completed 3 Dec, 2016 8:51:18 PM

如果在应用依赖项时有任何错误，请纠正我。谢谢

【问题讨论】：

请考虑评论您的反对意见？

标签： scala apache-spark glassfish sbt

【解决方案1】：

依赖项发布链中可能存在错误。一种选择是排除损坏的依赖项 - 将 exclude("org.glassfish.hk2", "hk2-utils") exclude("org.glassfish.hk2", "hk2-locator") exclude("javax.validation", "validation-api") 添加到引入这些依赖项的任何 Spark 依赖项中 - 并显式添加这三个依赖项的工作版本。

另外，我看不到org.glassfish.hk2#hk2-utils 或org.glassfish.hk2#hk2-locator 的版本2.22.2。

【讨论】：

如何举报这个 MavenCentral？不仅如此，还有一些在运行时出现的损坏的依赖关系，例如 fastxml-jackson

【解决方案2】：

在组织级别排除（如 How to exclude commons logging dependency of spring with ivy? 中的建议）对我有帮助。

我不得不使用这样的东西：

<dependency org="org.glassfish.jersey.core" name="jersey-client" rev="2.3" conf="binaries->default">
    <exclude  org="org.glassfish.hk2"/>
</dependency>
<dependency org="org.glassfish.hk2" name="hk2-utils" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-locator" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-api" rev="2.2.0-b14" conf="binaries->default"/>

【讨论】：