已创建 Hive 表，但映射任务失败并出现异常答案

【问题标题】：Hive table created, however map task is failing with exception已创建 Hive 表，但映射任务失败并出现异常
【发布时间】：2016-03-01 16:42:55
【问题描述】：

我在 Hive（CDH 4.6 上的 hive-0.10.0）中创建了一个推文表，尝试执行查询时遇到以下错误

hive> select * from tweets where id='442990122795929600';
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201404071250_0010, Tracking URL = http://swapnil:50030/jobdetails.jsp?jobid=job_201404071250_0010
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_201404071250_0010
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2014-04-08 00:45:24,755 Stage-1 map = 0%,  reduce = 0%
2014-04-08 00:46:25,892 Stage-1 map = 0%,  reduce = 0%
2014-04-08 00:47:06,691 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201404071250_0010 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://swapnil:50030/jobdetails.jsp?jobid=job_201404071250_0010
Examining task ID: task_201404071250_0010_m_000002 (and more) from job job_201404071250_0010

Task with the most failures(4): 
-----
Task ID:
  task_201404071250_0010_m_000000

URL:
  http://swapnil:50030/taskdetails.jsp?jobid=job_201404071250_0010&tipid=task_201404071250_0010_m_000000
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched: 
Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

我尝试将 jar "hive-contrib-0.10.0-cdh4.6.0.jar" 添加到类路径，但没有成功。

谁能帮忙解决这个问题？

【问题讨论】：

您正在处理什么样的数据？是 JSON/AVRO 文件还是纯文本文件？
我正在处理twitter的JSON数据
可能是您的数据和架构不匹配。你能分享你的配置单元表模式和 JSON 数据文件吗？还有你使用的是哪个 JSON Serde？span>

标签： java hadoop hive cloudera-cdh

【解决方案1】：

HIVE-SITE.XML 不在类路径中，hive 指的是 hive-default.xml。因此，hive-metastore 服务没有运行。

参考修改hadoop和hive的环境配置文件

hive-site.xml not found on classpath

【讨论】：

【解决方案2】：

您能否检查HIVE 中的hive.exec.max.created.files 变量 - 当您的Map/Reducer 创建的文件数超过100000 的max 文件时（默认值） - 它会终止工作另请检查您的 hive.log 以了解有关您的问题的更多信息。

【讨论】：