OOZIE：PIG 作业执行失败答案

【问题标题】：OOZIE:PIG job execution getting failedOOZIE：PIG 作业执行失败
【发布时间】：2016-03-02 16:48:41
【问题描述】：

我是OOZIE 的新手，我正在尝试使用OOZIE 工作流运行PIG 脚本。下面是名为first.pig的猪脚本：

A = LOAD '/user/jas/pigip' USING PigStorage(',');
B = FOREACH A GENERATE $0;
STORE B INTO '/user/jas/pigop';

下面是workflow.xml：

<workflow-app xmlns="uri:oozie:workflow:0.2" name="PIGACTION">
 <start to="pig-node"/>
 <action name="pig-node">
    <pig>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <script>first.pig</script>
    </pig>
  <ok to="end"/>
  <error to="fail"/>
 </action>
   <kill name="fail">
   <message>Pig Script failed!!!</message>
   </kill>
   <end name="end"/>
</workflow-app>

下面是job.properties：

nameNode=hdfs://localhost:8020
jobTracker=localhost:8021
queueName=default
oozie.libpath=/user/oozie/shared/lib
oozie.wf.application.path=${nameNode}/user/my/pigact

用于运行工作流：

1) 我在/user/my/pigact 位置上传了first.pig 和workflow.xml

2) 输入文件（一个简单的 CSV）上传到路径/user/jas/pigip

为了运行作业，下面是我使用的命令：

oozie job -oozie http://localhost:11000/oozie -config job.properties -auth SIMPLE -run

作业已提交，然后被终止。下面是Job Log：

2016-03-02 11:35:01,463  INFO ActionStartXCommand:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@:start:] Start action [0000009-160301223816814-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-02 11:35:01,463  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@:start:] [***0000009-160301223816814-oozie-oozi-W@:start:***]Action status=DONE
2016-03-02 11:35:01,464  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@:start:] [***0000009-160301223816814-oozie-oozi-W@:start:***]Action updated in DB!
2016-03-02 11:35:01,524  INFO ActionStartXCommand:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] Start action [0000009-160301223816814-oozie-oozi-W@pig-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-02 11:35:02,411  WARN PigActionExecutor:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] credentials is null for the action
2016-03-02 11:35:04,170  INFO PigActionExecutor:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] checking action, external ID [job_201603012236_0015] status [RUNNING]
2016-03-02 11:35:04,293  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] [***0000009-160301223816814-oozie-oozi-W@pig-node***]Action status=RUNNING
2016-03-02 11:35:04,295  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] [***0000009-160301223816814-oozie-oozi-W@pig-node***]Action updated in DB!
2016-03-02 11:35:12,210  INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] callback for action [0000009-160301223816814-oozie-oozi-W@pig-node]
2016-03-02 11:35:12,409  INFO PigActionExecutor:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] action completed, external ID [job_201603012236_0015]
2016-03-02 11:35:12,434  WARN PigActionExecutor:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
2016-03-02 11:35:12,705  INFO ActionEndXCommand:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@pig-node] ERROR is considered as FAILED for SLA
2016-03-02 11:35:12,794  INFO ActionStartXCommand:539 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@fail] Start action [0000009-160301223816814-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-02 11:35:12,795  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@fail] [***0000009-160301223816814-oozie-oozi-W@fail***]Action status=DONE
2016-03-02 11:35:12,795  WARN ActionStartXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814-oozie-oozi-W@fail] [***0000009-160301223816814-oozie-oozi-W@fail***]Action updated in DB!
2016-03-02 11:35:12,904  WARN CoordActionUpdateXCommand:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

请提出问题所在，我无法确定问题所在。

编辑

捕捉快照：

第 3 张图片（突出显示）中的控制台 URL 正在显示第 4 张图片。请提出建议。

这是我得到的日志：

>>> Invoking Pig command line now >>>


Run pig script using PigRunner.run() for Pig version 0.8+
Apache Pig version 0.10.0-cdh4.1.1 (rexported) 
compiled Oct 16 2012, 10:15:57

Run pig script using PigRunner.run() for Pig version 0.8+
1196 [main] INFO  org.apache.pig.Main  - Apache Pig version 0.10.0-cdh4.1.1 (rexported) compiled Oct 16 2012, 10:15:57
1199 [main] INFO  org.apache.pig.Main  - Logging error messages to: /var/lib/hadoop-hdfs/cache/mapred/mapred/local/taskTracker/training/jobcache/job_201603012236_0030/attempt_201603012236_0030_m_000000_0/work/pig-job_201603012236_0030.log
1463 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine  - Connecting to hadoop file system at: hdfs://localhost:8020
1472 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine  - Connecting to map-reduce job tracker at: localhost:8021
1674 [main] ERROR org.apache.pig.tools.grunt.Grunt  - ERROR 2998: Unhandled internal error. name

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

Oozie Launcher failed, finishing Hadoop job gracefully


Oozie Launcher ends


stderr logs

Details at logfile: /var/lib/hadoop-hdfs/cache/mapred/mapred/local/taskTracker/training/jobcache/job_201603012236_0030/attempt_201603012236_0030_m_000000_0/work/pig-job_201603012236_0030.log

Pig logfile dump:

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. name

java.lang.NoSuchFieldError: name
    at org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
    at org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:198)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:166)
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1594)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1545)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:545)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:430)
    at org.apache.pig.PigRunner.run(PigRunner.java:49)
    at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:282)
    at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:222)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
================================================================================
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

【问题讨论】：

这可以帮助你stackoverflow.com/questions/15336923/…
我之前检查过这篇文章，但它没有用：我可以使用MapReduce 运行OOZIE 工作流，因此启动OOZIE 服务器应该不是问题。对于PIG 和HIVE，我都面临这个问题。请提出建议。

标签： hadoop mapreduce apache-pig oozie cloudera-cdh

【解决方案1】：

查看QueryParserStringStream.java，它扩展了 ANTLRStringStream。现在如果缺少 antlr，你应该得到 ClassNotFoundError 而不是 NoSuchFiledError。我能想到的唯一可能原因是运行时类路径中存在不同版本的 antlr，这导致了此错误。

【讨论】：

从 oozie 共享库（来自 HDFS）中删除重复的 jar 文件并使用正确版本的 antlr jar 文件为我解决了这个问题。再次感谢！！！
我可以知道您删除了哪个特定的jar文件吗？

【解决方案2】：

您看到的错误是 oozie 启动器映射器作业的失败结果，该作业携带猪脚本来收集资源并在您的集群上实际运行它。它基本上模仿了您的命令行 grunt 功能。

如果您有可用的 oozie Web 控制台，您可以单击工作流作业上的已终止操作，这将显示作业 URL。作业 URL 实际上指向您的 oozie 启动器映射器作业，您可以检查其日志以了解问题所在。

【讨论】：

当您单击已终止的工作流时，您应该能够看到错误输出的实际猪操作。如果你点击errored out pig action，你应该可以看到Console URL，这将打开启动器作业
我看不到Console URL 标签，我正在使用cloudera 4.1.1。另外，我已经在本地运行了 Pig 脚本，它工作正常，因此 Pig 脚本没有问题。当我尝试使用 HIVE 作业运行 OOZIE 工作流时，我遇到了同样的错误，bur MapReduce 作业在 OOZIE 工作流中运行良好。请提出建议。
你能贴一张被杀死的工作的截图吗？单击失败的操作后，您应该能够看到三个选项卡
在这里注意到了类似的东西 - stackoverflow.com/questions/17179200/… 你能检查一下你是否在同一台机器上启用了 hive，很可能是？
是的，我确实在同一系统中启用了 HIVE。如何禁用蜂巢？我尝试了echo $HIVE_HOME，但终端中没有显示任何内容，但是 HIVE 已启用并且在我的系统中运行良好。然后我尝试了export HIVE_HOME= 并重新运行 oozie 作业，但我仍然遇到同样的错误。请告诉我如何在我的系统中禁用 HIVE。

【解决方案3】：

执行 pig 脚本时出错。这是日志中的错误：

2016-03-02 11:35:12,434 WARN PigActionExecutor:542 - USER[training] GROUP[-] TOKEN[] APP[PIGACTION] JOB[0000009-160301223816814-oozie-oozi-W] ACTION[0000009-160301223816814 -oozie-oozi-W@pig-node] 启动器错误，原因：主类 [org.apache.oozie.action.hadoop.PigMain]，退出代码 [2]

我在你的脚本中没有发现任何明显的问题，你能在 MapReduce 模式下运行脚本并检查是否有任何错误吗？

pig -x mapreduce first.pig

【讨论】：

在您的设置中是否启用了 tez 作为 PIG 的默认执行引擎？
No tez 尚未启用。是否需要解决此问题？为了添加更多信息，我在 Cloudera 集群上运行它，并且我在 HIVE 作业中也遇到了同样的错误。