为什么构建 Spark 源会给出“object sbt is not a member of package com.typesafe”？答案

【问题标题】：Why does building Spark sources give "object sbt is not a member of package com.typesafe"?为什么构建 Spark 源会给出“object sbt is not a member of package com.typesafe”？
【发布时间】：2014-07-29 09:05:24
【问题描述】：

我尝试在 Windows 上使用带有 sbt 插件的 IntelliJ IDEA 编译 https://github.com/apache/spark 项目。

我遇到了一个关于 sbt 的错误。由于我对 sbt 不熟悉，所以不知道如何解决。

错误信息如下：

[info] Loading project definition from F:\codeReading\sbtt\spark-master\project
[info] Compiling 3 Scala sources to F:\codeReading\sbtt\spark-master\project\target\scala-2.10\sbt-0.13\classes...
[error] F:\codeReading\sbtt\spark-master\project\SparkBuild.scala:26: object sbt is not a member of package com.typesafe
[error] import com.typesafe.sbt.pom.{PomBuild, SbtPomKeys}
[error]                     ^
[error] F:\codeReading\sbtt\spark-master\project\SparkBuild.scala:51: not found: type PomBuild
[error] object SparkBuild extends PomBuild {
[error]                           ^
[error] F:\codeReading\sbtt\spark-master\project\SparkBuild.scala:118: not found: value SbtPomKeys
[error]     otherResolvers <<= SbtPomKeys.mvnLocalRepository(dotM2 => Seq(Resolver.file("dotM2", dotM2))),
[error]                        ^
[error] F:\codeReading\sbtt\spark-master\project\SparkBuild.scala:178: value projectDefinitions is not a member of AnyRef
[error]     super.projectDefinitions(baseDirectory).map { x =>
[error]           ^
[error] four errors found
[error] (plugins/compile:compile) Compilation failed

【问题讨论】：

你是如何在IDEA中执行编译的？这是在将项目导入 IDEA 时吗？
我从 github/apache/spark 下载了 spark-master.zip，解压，然后用 sbt 而不是 maven 导入解压后的项目。我在 Intellij-idea（Windows XP 操作系统）中的操作是：文件->导入项目->选择 spark-master 目录->选择“从外部模型导入项目”->选择“SBT 项目”->下一步->完成
是的，我使用IDEA导入了项目。

标签： intellij-idea sbt apache-spark

【解决方案1】：

Spark 是使用 Maven 构建的。 SBT 构建只是为了方便。将其作为 Maven 项目导入会得到更好的结果。

【讨论】：

你是对的。我尝试使用 maven 编译 Spark，并且成功了。
哎呀，我错过了那个开关。他们什么时候迁移到 Maven 的？我记得 1.0.0 仍然使用 sbt 和 maven。谢谢。
我用maven导入的版本是1.0.0。我在 SBT 的 1.0.0 和 1.0.1 中都失败了。
还有一个从 Maven 导出的 SBT 构建，请注意。我认为这甚至可能是 1.0 之后的版本，但它已经在 master 中运行了几个星期。我假设 OP 是从 master 构建的。

【解决方案2】：

看起来 IDEA 不喜欢 Spark 构建定义用于 sbt-pom-reader 的 git 项目的项目引用。

当我在克隆项目中运行sbt 时，它出现了：

➜  spark git:(master) ✗ xsbt
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Loading project definition from /Users/jacek/oss/spark/project/project
[info] Loading project definition from /Users/jacek/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project
[warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
[info] Loading project definition from /Users/jacek/oss/spark/project
[info] Set current project to spark-parent (in build file:/Users/jacek/oss/spark/)
>

访问定义了引用的plugins项目时，可以看到对sbt-pom-reader的git项目的项目引用：

> reload plugins
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Loading project definition from /Users/jacek/oss/spark/project/project
[info] Updating {file:/Users/jacek/oss/spark/project/project/}project-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Loading project definition from /Users/jacek/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project
[warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`).
[info] Updating {file:/Users/jacek/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project/}sbt-pom-reader-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Loading project definition from /Users/jacek/oss/spark/project
> projects
[info] In file:/Users/jacek/oss/spark/project/
[info]   * plugins
[info]     spark-style
[info] In https://github.com/ScrapCodes/sbt-pom-reader.git
[info]     sbt-pom-reader

一个解决方案可能是执行sbt gen-idea 来为IDEA 生成项目文件。不过，这只是一个猜测。

【讨论】：