使用 importtsv 将文本文件导入 HBase

Question

问题：

我正在尝试使用文档中的 this 示例将数据从文本文件导入 HBase 数据库。

配置

HBase 版本：hbase-0.94.4.

我的安装是Standalone（非分布式或伪分布式）。

重现错误

最初它在 HDFS 上创建 StoreFiles，并使用名称为 datatsv 的空表，使用：

HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop \
jar /usr/local/hbase/hbase-0.94.4.jar importtsv \
-Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2 \
-Dimporttsv.bulk.output=/user/vm1/outputfile  datatsv /user/vm1/myfile.txt

然后，CompleteBulkLoad 方法将数据从 StoreFile 加载到表中：

hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles \
/user/vm1/outputfile datatsv

下面的命令与上面的输出相同：

HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop \
jar /usr/local/hbase/hbase-0.94.4.jar completebulkload \
/user/vm1/outputfile datatsv

任一命令的错误消息是：

ERROR mapreduce.LoadIncrementalHFiles: Unexpected execution exception during splitting
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
    at java.util.concurrent.FutureTask.get(FutureTask.java:111)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:333)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getEffectiveTableName(SchemaMetrics.java:607)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getInstance(SchemaMetrics.java:333)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:185)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:441)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:419)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:410)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:51)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:344)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    ... 10 more
Caused by: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getEffectiveTableName(SchemaMetrics.java:607)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getInstance(SchemaMetrics.java:333)
    at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:185)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:441)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:419)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:410)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
    at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

问题

我错过了什么吗？

搜索了邮件列表，在空闲节点上询问，并没有找到答案。还尝试使用以下方法设置 hbase 的属性（并重新启动它）： hbase.metrics.showTableName 到 false

任何帮助将不胜感激！ 谢谢！

Answer 1

不久前，当我尝试使用 Java API 批量加载数据时，我遇到了一个非常相似的问题。我开了一个线程here。我必须在我的客户端应用程序中指定这一行，这解决了这个问题：SchemaMetrics.configureGlobally(conf)。现在，您不使用 Java API，这行所做的只是在 .xml 配置中设置一个属性，因此您应该能够手动设置它。我会试试这个：hbase.metrics.showTableName，默认情况下应该已经是true，但我会尝试将它显式设置为true。目前我正在研究 HBase 0.90（Hortonworks HDP 1.1 附带的那个），我什至不必再使用SchemaMetrics.configureGlobally(conf)。也许这是 HBase 0.94 中的一个错误（或只是一种新方法）？

Answer 2

我询问的工具不再工作。 因此，我编写了一些 Java 代码，使用 HBase API 使用 MapReduce 进行批量加载。

Github 链接：HBase Bulk Load