【发布时间】:2019-01-21 16:08:55
【问题描述】:
我开始学习 Solr,我正在尝试使用 Solr 和 Spark 编写 WordCount 示例。但我有问题,可能是导入或依赖项。您可以在下面查看我的代码..
我的依赖:
<groupId>com.lucidworks.spark</groupId>
<artifactId>spark-solr</artifactId>
<version>2.1.0</version>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
<version>7.6.0</version>
我的代码:
object Solr extends SparkApp.RDDProcessor {
def getName: String = "query-solr-benchmark"
def getOptions: Array[Option] = {
Array(
Option.builder()
.argName("QUERY")
.longOpt("query")
.hasArg
.required(false)
.desc("URL encoded Solr query to send to Solr")
.build()
)
}
def run(conf: SparkConf, cli: CommandLine): Int = {
val zkHost = cli.getOptionValue("zkHost", "localhost:9983")
val collection = cli.getOptionValue("collection", "collection1")
val queryStr = cli.getOptionValue("query", "*:*")
val rows = cli.getOptionValue("rows", "1000").toInt
val splitsPerShard = cli.getOptionValue("splitsPerShard", "3").toInt
val splitField = cli.getOptionValue("splitField", "_version_")
val sc = new SparkContext(conf)
val solrQuery: SolrQuery = new SolrQuery(queryStr)
val fields = cli.getOptionValue("fields", "")
if (!fields.isEmpty)
fields.split(",").foreach(solrQuery.addField)
solrQuery.addSort(new SolrQuery.SortClause("id", "asc"))
solrQuery.setRows(rows)
val solrRDD: SolrRDD = new SolrRDD(zkHost, collection, sc)
var startMs: Long = System.currentTimeMillis
var count = solrRDD.query(solrQuery).splitField(splitField).splitsPerShard(splitsPerShard).count()
var tookMs: Long = System.currentTimeMillis - startMs
println(s"\nTook $tookMs ms read $count docs using queryShards with $splitsPerShard splits")
// IMPORTANT: reload the collection to flush caches
println(s"\nReloading collection $collection to flush caches!\n")
val cloudSolrClient = SolrSupport.getCachedCloudClient(zkHost)
val req = CollectionAdminRequest.reloadCollection(collection)
cloudSolrClient.request(req)
startMs = System.currentTimeMillis
count = solrRDD.query(solrQuery).count()
tookMs = System.currentTimeMillis - startMs
println(s"\nTook $tookMs ms read $count docs using queryShards")
sc.stop()
}
}
我的问题是builder() 是红色的,我无法运行我的代码。有谁知道我错过了什么?
错误:值生成器不是对象 org.apache.commons.cli.Option 的成员
【问题讨论】:
-
请添加实际编译器错误的文本和行(不是IDE警告)
-
没用
标签: scala intellij-idea solr solrj