Hbase Region 服务器关闭答案

【问题标题】：Hbase Region server shutdownHbase Region 服务器关闭
【发布时间】：2023-03-26 08:39:01
【问题描述】：

我是 Mohanselvakumar，是 Hbase 系统的新手，请求在我的一台 hbase 服务器上帮助我完成以下工作。

场景是我设置了 10 个节点集群和 2 个名称节点（一个主节点和一个辅助节点）。

在这 10 个节点上，我在所有节点上运行 Datanode 以及 Regionserver 服务。

现在问题出在10个节点之外，其中一个节点因为以下异常而不断宕机；

Frequent Error No:01

2013-01-25 12:10:51,918 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server serverName=DC1AuthDFSC1D1.cidr.gov.in,6020,1359093222948, load=(requests=10, regions=876, usedHeap=1925, maxHeap=11897): regionserver:6020-0x3c13d364d92db2-0x3c13d364d92db2 regionserver:6020-0x3c13d364d92db2-0x3c13d364d92db2 received expired from ZooKeeper, aborting
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
    at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:328)
    at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:246)
    at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
2013-01-25 12:10:51,931 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: request=0.0, regions=876, stores=992, storefiles=2380, storefileIndexSize=769, memstoreSize=19, compactionQueueSize=0, flushQueueSize=0, usedHeap=2043, maxHeap=11897, blockCacheSize=808359784, blockCacheFree=1686753176, blockCacheCount=11005, blockCacheHitCount=80692, blockCacheMissCount=149039, blockCacheEvictedCount=0, blockCacheHitRatio=35, blockCacheHitCachingRatio=87
2013-01-25 12:10:51,931 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: regionserver:6020-0x3c13d364d92db2-0x3c13d364d92db2 regionserver:6020-0x3c13d364d92db2-0x3c13d364d92db2 received expired from ZooKeeper, aborting
2013-01-25 12:10:51,932 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2013-01-25 12:10:54,703 INFO org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
2013-01-25 12:10:58,660 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: regionserver6020.compactor exiting
2013-01-25 12:11:00,866 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: regionserver6020.cacheFlusher exiting
2013-01-25 12:11:02,647 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 6020
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 5 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 3 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 6020
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 6 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 6020: exiting
2013-01-25 12:11:02,648 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2013-01-25 12:11:02,649 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 6020: exiting
2013-01-25 12:11:02,649 INFO org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 0 on 6020: exiting

Frequent Error No:02

    2013-01-25 12:11:19,106 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver6020 closing leases
    2013-01-25 12:11:19,106 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver6020 closed leases
    2013-01-25 12:11:19,136 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13c13d323792d99
    2013-01-25 12:11:19,138 INFO org.apache.zookeeper.ZooKeeper: Session: 0x13c13d323792d99 closed
    2013-01-25 12:11:19,138 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
    2013-01-25 12:11:19,143 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver6020 exiting
    2013-01-25 12:11:19,152 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-14,5,main]
    2013-01-25 12:11:19,152 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Shutdown hook
    2013-01-25 12:11:19,152 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown hook thread.
    2013-01-25 12:11:19,153 ERROR org.apache.hadoop.hdfs.DFSClient: Exception closing file

【问题讨论】：

感谢您对上述查询的帮助！！！

标签： hadoop hbase

【解决方案1】：

Zookeeper 过期可能由于节点内存不足、GC 太长等原因而发生。请参阅第 12.9.2.7 节 here。

【讨论】：

【解决方案2】：

您可以像这样在 hbase-site.xml 中添加会话超时

      <property>
        <name>zookeeper.session.timeout</name>
        <value>60000</value>
      </property>

跳过这个帮助

【讨论】：