【问题标题】:HBase Import commandHBase 导入命令
【发布时间】:2014-08-15 23:12:06
【问题描述】:

我们目前正在从 CDH3u4 迁移到 CDH5。我们创建了新集群并复制了所有数据。多亏了 Cloudera 经理,一切都很顺利。但是我们在将数据从 HBase 0.90.6 迁移到 HBase 0.96.1.1 时遇到了问题。

我尝试使用 HBase (https://hbase.apache.org/book/ops_mgt.html#export) 的导出/导入功能迁移数据。我已成功导出数据并将它们复制到新服务器 (discp)。当我在目标集群上使用命令时:

hbase -Dhbase.import.version=0.90 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3

作业已成功完成,但未加载任何数据(表 ip 仍为空白):

14/06/25 09:04:58 INFO mapreduce.Job: Job job_1403615212297_0014 running in uber mode : false
14/06/25 09:04:58 INFO mapreduce.Job: map 0% reduce 0%
14/06/25 09:05:08 INFO mapreduce.Job: map 7% reduce 0%
14/06/25 09:05:11 INFO mapreduce.Job: map 43% reduce 0%
14/06/25 09:05:16 INFO mapreduce.Job: map 45% reduce 0%
14/06/25 09:05:18 INFO mapreduce.Job: map 50% reduce 0%
14/06/25 09:05:20 INFO mapreduce.Job: map 55% reduce 0%
14/06/25 09:05:21 INFO mapreduce.Job: map 57% reduce 0%
14/06/25 09:05:22 INFO mapreduce.Job: map 80% reduce 0%
14/06/25 09:05:23 INFO mapreduce.Job: map 86% reduce 0%
14/06/25 09:05:25 INFO mapreduce.Job: map 91% reduce 0%
14/06/25 09:05:26 INFO mapreduce.Job: map 98% reduce 0%
14/06/25 09:05:28 INFO mapreduce.Job: map 100% reduce 0%
14/06/25 09:05:28 INFO mapreduce.Job: Job job_1403615212297_0014 completed successfully
14/06/25 09:05:28 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=5172058
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=5452414893
HDFS: Number of bytes written=0
HDFS: Number of read operations=132
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Job Counters
Launched map tasks=44
Data-local map tasks=44
Total time spent by all maps in occupied slots (ms)=410004
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=410004
Total vcore-seconds taken by all map tasks=410004
Total megabyte-seconds taken by all map tasks=419844096
Map-Reduce Framework
Map input records=9964456
Map output records=0
Input split bytes=5720
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=7648
CPU time spent (ms)=117230
Physical memory (bytes) snapshot=17097363456
Virtual memory (bytes) snapshot=68115570688
Total committed heap usage (bytes)=26497384448
File Input Format Counters
Bytes Read=5452409173
File Output Format Counters
Bytes Written=0

当我查看日志时,这里没有错误。

如有任何想法,我将不胜感激,非常感谢!

【问题讨论】:

    标签: hadoop import export hbase


    【解决方案1】:

    看来问题出在命令上:

    hbase -Dhbase.import.version=0.90 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3
    

    当我这样修改它时,整个工作都很顺利:

    hbase -Dhbase.import.version=0.94 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3
    

    认为 import.version=0.90 不受支持。

    【讨论】:

    • 有同样的问题。不幸的是,您的解决方案对我不起作用。
    【解决方案2】:

    我有同样的问题,但你的解决方案对我不起作用。

    我尝试了很多时间,发现一旦我在运行导入任务之前禁用表。出现“regionserver not online”的错误,但在任务运行期间,我启用了表。导入任务顺利结束,新数据加载完毕!!!

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2012-09-01
      • 1970-01-01
      • 1970-01-01
      • 2019-11-10
      • 2017-12-23
      • 1970-01-01
      相关资源
      最近更新 更多