【发布时间】：2019-01-18 02:18:17
【问题描述】：
我尝试使用 yarn 来运行 OLAP 和批量加载。版本是 janusgrapg-0.3.2,spark-2.2.1
为了在 janusgraph 中启动 spark-yarn，我将 spark jar 复制到 $JANUSGRAPH_HOME/lib 中。
hadoop-graphson.properties：
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONInputFormat
gremlin.hadoop.graphWriter=org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONOutputFormat
gremlin.hadoop.inputLocation=data/tinkerpop-modern.json
gremlin.hadoop.outputLocation=output
gremlin.hadoop.jarsInDistributedCache=true
gremlin.vertexProgram=org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankVertexProgram
giraph.minWorkers=2
giraph.maxWorkers=2
spark.master=yarn
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator
gremlin 控制台中的命令：
graph = GraphFactory.open('/Users/lwh/dev/janusgraph-0.3.2-SNAPSHOT-hadoop2/conf/hadoop-graph/hadoop-graphson.properties')
blvp = BulkLoaderVertexProgram.build().bulkLoader(OneTimeBulkLoader).writeGraph('/Users/lwh/dev/janusgraph-0.3.2-SNAPSHOT-hadoop2/conf/janusgraph-cql-es.properties').create(graph);graph.compute(SparkGraphComputer).program(blvp).submit().get()
纱线中的结果：
Application application_1547728549460_0008 failed 2 times due to AM Container for appattempt_1547728549460_0008_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://lwhdeMacBook-Pro.local:8088/cluster/app/application_1547728549460_0008Then, click on links to logs of each attempt.
Diagnostics: Resource file:/Users/lwh/.sparkStaging/application_1547728549460_0008/__spark_libs__7746382379788050915.zip changed on src filesystem (expected 1547732288000, was 1547732288958
java.io.IOException: Resource file:/Users/lwh/.sparkStaging/application_1547728549460_0008/__spark_libs__7746382379788050915.zip changed on src filesystem (expected 1547732288000, was 1547732288958
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:255)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:361)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:358)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Failing this attempt. Failing the application.
【问题讨论】：
标签： janusgraph