pig map-reduce 作业在完成 33% 后失败答案

【问题标题】：pig map-reduce job fails after completing 33%pig map-reduce 作业在完成 33% 后失败
【发布时间】：2016-04-04 19:10:47
【问题描述】：

我在 apache pig 中运行 group by 子句，它正在创建 map reduce 作业，在 1/3 完成后失败。无论如何我可以解决这个问题，因为日志没有给出任何失败的原因。我在看什么。
1. 找出确切错误的方法（即内存错误、数据类型错误等）
2. 任何使日志更详细的方法，以便在屏幕上写入更多错误消息。

2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1459707650021_0003
2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a,b
2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],b[2,4] C:  R:
2016-04-03 22:59:40,258 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2016-04-03 22:59:40,258 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:10,428 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 33% complete
2016-04-03 23:00:10,428 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:15,435 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2016-04-03 23:00:15,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1459707650021_0003 has failed! Stop running all dependent jobs
2016-04-03 23:00:15,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2016-04-03 23:00:15,508 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://compute.internal:8188/ws/v1/timeline/
2016-04-03 23:00:15,511 [main] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at compute.internal/8050
2016-04-03 23:00:15,515 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server

【问题讨论】：

Pig 会有一个与作业相关联的日志文件。该名称将在运行此作业时出现。检查那里你会得到失败的原因。

标签： hadoop mapreduce apache-pig

【解决方案1】：

有一个 JobId 与您提交给 map reduce 框架的每个作业相关联。复制该 JobId ，粘贴并在您的工作跟踪器中搜索。例如，CDH3 by cluster cloudera 在浏览器中设置了作业跟踪器，您可以在其中查看所有作业的详细信息。

【讨论】：