【发布时间】：2018-10-06 04:56:34
【问题描述】：
在 HBASE 中创建表时出错。 “错误：java.io.IOException：表命名空间管理器尚未准备好，请尝试 稍后再来。”
hcbk -fix 显示 ERROR: hbase:meta is not found on any region.
在重新启动 hbase shell 会话后出现错误启动期间主日志中没有报告错误。 上次 Hbase 会话正常关闭，但 zookeeper 未正常关闭（怀疑这是元表损坏的原因）。
我可以列出之前创建的表
hbase(main):001:0> list
TABLE
IDX_STOCK_SYMBOL
Patient
STOCK_SYMBOL
STOCK_SYMBOL_BKP
SYSTEM.CATALOG
SYSTEM.FUNCTION
SYSTEM.SEQUENCE
SYSTEM.STATS
8 row(s) in 1.7930 seconds
创建一个名为 custmaster 的表
hbase(main):002:0> create 'custmaster', 'customer'
ERROR: java.io.IOException: Table Namespace Manager not ready yet, try
again later
at org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:3179)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1735)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1774)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40470)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
解决方法：运行 hbck 以识别不一致
[hduser@master ~]$ hbase hbck
>Version: 0.98.4-hadoop2
>Number of live region servers: 2
>Number of dead region servers: 0
>Master: master,60000,1538793456542
>Number of backup masters: 0
>Average load: 0.0
>Number of requests: 11
>Number of regions: 0
>Number of regions in transition: 1
>
>ERROR: META region or some of its attributes are null.
>ERROR: hbase:meta is not found on any region.
>ERROR: hbase:meta table is not consistent. Run HBCK with proper fix options to fix hbase:meta inconsistency. Exiting...
.
.
.
>Summary: >
>3 inconsistencies detected.
>Status: INCONSISTENT
运行 hbck wih-details 选项以识别所涉及的表
[hduser@master ~]$ hbase hbck -details
>ERROR: META region or some of its attributes are null.
>ERROR: hbase:meta is not found on any region.
>ERROR: hbase:meta table is not consistent. Run HBCK with proper fix options to fix hbase:meta inconsistency. Exiting...
>Summary:
>3 inconsistencies detected.
>Status: INCONSISTENT
-details 的输出清楚地表明在任何区域都找不到元数据。
尝试运行命令 hbase hbck -fixMeta 但返回的结果与上面相同 因此尝试了 hbase hbck -fix
此命令运行了一段时间，提示 “尝试修复 hbase:meta.. 的问题” 并导致以下错误
[hduser@master ~]$ hbase hbck -fix
Version: 0.98.4-hadoop2
Number of live region servers: 2
Number of dead region servers: 0
Master: master,60000,1538793456542
Number of backup masters: 0
Average load: 0.0
Number of requests: 19
Number of regions: 0
Number of regions in transition: 1
ERROR: META region or some of its attributes are null.
ERROR: hbase:meta is not found on any region.
Trying to fix a problem with hbase:meta..
2018-10-06 09:01:03,424 INFO [main] client.HConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2018-10-06 09:01:03,425 INFO [main] client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x166473bbe720005
2018-10-06 09:01:03,432 INFO [main] zookeeper.ZooKeeper: Session: 0x166473bbe720005 closed
2018-10-06 09:01:03,432 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
Sat Oct 06 08:52:13 IST 2018, org.apache.hadoop.hbase.client.RpcRetryingCaller@18920cc, org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.PleaseHoldException): org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2416)
at org.apache.hadoop.hbase.master.HMaster.assignRegion(HMaster.java:2472)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40456)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Sat Oct 06 08:52:13 IST 2018, org.apache.hadoop.hbase.client.RpcRetryingCaller@18920cc, org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.PleaseHoldException): org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2416)
at org.apache.hadoop.hbase.master.HMaster.assignRegion(HMaster.java:2472)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40456)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
请帮我解决这个问题？ 提前致谢！！
【问题讨论】：