【发布时间】：2015-07-07 04:05:14
【问题描述】：
我使用的是 Apache Storm 0.9.4。 我设置了一个 5 个节点的集群，它工作正常。 （实际上 5 个主管在 5 个不同的物理节点上作为 docker 容器工作。）
我的环境在这里
$cat /etc/redhat-release
CentOS release 6.6 (Final)
$docker -v
Docker version 1.4.1, build 5bc2ff8/1.4.1
$java -version
java version "1.7.0_79" OpenJDK Runtime Environment
(rhel-2.5.5.1.el6_6-x86_64 u79-b14) OpenJDK 64-Bit Server VM (build
24.79-b02, mixed mode)
接下来，我创建一个 maven 项目以使用 Netbeans 提交拓扑。
有1个topology和1个spout。（很简单）
- MyTopology.java
- LoggerSpout.java
MyTopology 在没有 GSON 的情况下可以正常工作。
但是，在我在nextTuple() 函数中添加Gson gson = new Gson(); 之后，
我得到了 java.lang.NoClassDefFoundError: com/google/gson/Gson。
这就是我所做的
-
1. mvn clean -> mvn compile -> mvn package -> 构建成功
2.storm jar ~/Desktop/teststorm/target/teststorm-1.0.jar jp.soushi.teststorm.MyTopology -> 提交即可。
Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.4 -Dstorm.log.dir=/usr/local/apache-storm-0.9.4/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.4/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.4/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.4/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.4/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.4/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.4/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.4/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.4/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.4/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.4/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.4/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.4/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.4/lib/disruptor-2.10.1.jar:/usr/local/apache-storm-0.9.4/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.4/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.4/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.4/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.4/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.4/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.4/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.4/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.4/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.4/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.4/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.4/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.4/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.4/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.4/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.4/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.4/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.4/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.4/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.4/lib/storm-core-0.9.4.jar:/usr/local/apache-storm-0.9.4/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.4/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.4/lib/tools.macro-0.1.0.jar:/Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar:/Users/soushi/.storm:/usr/local/apache-storm-0.9.4/bin -Dstorm.jar=/Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar jp.soushi.teststorm.MyTopology
491 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
499 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar /Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar to assigned location: storm-local/nimbus/inbox/stormjar-b6added0-ffb8-4602-9d89-b567ed87d335.jar
509 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-b6added0-ffb8-4602-9d89-b567ed87d335.jar
509 [main] INFO backtype.storm.StormSubmitter - Submitting topology MyTopology in distributed mode with conf {"topology.workers":31,"topology.debug":true,"topology.max.spout.pending":5000}
766 [main] INFO backtype.storm.StormSubmitter - Finished submitting topology: MyTopology
- 我使用 Storm Web UI 检查了拓扑。过了一会儿，我得到了 NoClassDefFoundError。
- 我编辑了 nimbus.yaml，并添加了
java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib/java"。并确保 /usr/lib/java 可以在 docker 容器中访问。
这是错误日志（worker-6727.log）。
2015-04-28T10:05:18.497+0000 b.s.d.executor [ERROR]
java.lang.NoClassDefFoundError: com/google/gson/Gson
at jp.soushi.adcontex.SensorSpout.nextTuple(SensorSpout.java:43) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fn__4654$fn__4669$fn__4698.invoke(executor.clj:565) ~[storm-core-0.9.4.jar:0.9.4]
at backtype.storm.util$async_loop$fn__458.invoke(util.clj:463) ~[storm-core-0.9.4.jar:0.9.4]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: java.lang.ClassNotFoundException: com.google.gson.Gson
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_79]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_79]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_79]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_79]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_79]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) ~[na:1.7.0_79]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_79]
... 5 common frames omitted
2015-04-28T10:05:18.564+0000 b.s.util [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.4.jar:0.9.4]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$fn__5102$fn__5103.invoke(worker.clj:495) [storm-core-0.9.4.jar:0.9.4]
at backtype.storm.daemon.executor$mk_executor_data$fn__4555$fn__4556.invoke(executor.clj:240) [storm-core-0.9.4.jar:0.9.4]
at backtype.storm.util$async_loop$fn__458.invoke(util.clj:473) [storm-core-0.9.4.jar:0.9.4]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
- pom.xml 我尝试更改 gson 的作用域（运行时、提供、编译），但它仍然不起作用
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>jp.soushi</groupId>
<artifactId>teststorm</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>teststorm</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>jp.soushi.adcontex.AdContexTopology</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy</id>
<phase>install</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
</repository>
<repository>
<id>github-releases</id>
<url>http://oss.sonatype.org/content/repositories/github-releases/</url>
</repository>
<repository>
<id>clojars.org</id>
<url>http://clojars.org/repo</url>
</repository>
<repository>
<id>local-project-libraries</id>
<name>Local project libraries</name>
<url>file://${project.basedir}/lib</url>
<layout>default</layout>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>storm</groupId>
<artifactId>storm-lib</artifactId>
<version>0.9.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.3.1</version>
<type>jar</type>
<scope>runtime</scope>
</dependency>
</dependencies>
</project>
- MyTopology.java
package jp.soushi.teststorm;
import backtype.storm.Config;
import backtype.storm.StormSubmitter;
import backtype.storm.topology.TopologyBuilder;
public class MyTopology {
public static void main(String[] args) throws Exception {
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("Logs", new LoggerSpout(), 5);
Config conf = new Config();
conf.setDebug(true);
conf.setNumWorkers(31);
conf.setMaxSpoutPending(5000);
StormSubmitter.submitTopology( "MyTopology", conf, builder.createTopology() );
}
}
- LoggerSpout.java
package jp.soushi.teststorm;
import backtype.storm.Config;
import backtype.storm.spout.SpoutOutputCollector;
import backtype.storm.task.TopologyContext;
import backtype.storm.topology.OutputFieldsDeclarer;
import backtype.storm.topology.base.BaseRichSpout;
import backtype.storm.tuple.Fields;
import backtype.storm.tuple.Values;
import backtype.storm.utils.Utils;
import com.google.gson.Gson;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LoggerSpout extends BaseRichSpout {
public static Logger LOG = LoggerFactory.getLogger(LoggerSpout.class);
boolean _isDistributed;
SpoutOutputCollector _collector;
public LoggerSpout() {
this(true);
}
public LoggerSpout(boolean isDistributed) {
_isDistributed = isDistributed;
}
public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
_collector = collector;
}
public void close() {
}
public void nextTuple() {
Utils.sleep(100);
Gson gson = new Gson();
_collector.emit(new Values("Test"));
}
public void ack(Object msgId) {
}
public void fail(Object msgId) {
}
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("Logs"));
}
@Override
public Map<String, Object> getComponentConfiguration() {
if(!_isDistributed) {
Map<String, Object> ret = new HashMap<String, Object>();
ret.put(Config.TOPOLOGY_MAX_TASK_PARALLELISM, 1);
return ret;
} else {
return null;
}
}
}
我找到了这个答案。但是我不是很明白。 java.lang.NoClassDefFoundError: com/google/gson/Gson
我也阅读了这些答案。
- Getting java.lang.ClassNotFoundException: com.google.gson.Gson for maven project
- GSON is not being imported into the maven pproject
- http://www.ciiycode.com/0JSJ6PgUXWxq/javalangnoclassdeffounderror-comgooglegsongsonbuilder-error
我花了 3 天时间来解决这个问题……我不知道。 任何人都可以为此提供任何解决方案。在此先感谢。
【问题讨论】：
-
尝试从 pom.xml 中的 Gson 依赖项中删除
<scope>和<type>标签 -
非常感谢。我删除了类型元素。但是效果不好。
-
适用于所有有相同错误的人。我想出了正确的代码。如果您有相同的错误，请按照以下步骤操作。 1. 编辑 pom.xml 以删除 Neeraj Jain 评论后的
元素。 2. 更改 以进行编译。即 compile 按照 swapyonubuntu 的回答。 3. 在 이승진 的回答之后，将 maven-shade-plugin 添加到您的 pom.xml 文件中。我真的很感谢那些给出答案的人。我的拓扑现在运行良好。非常感谢。
标签： java maven gson noclassdeffounderror apache-storm