报错信息
2023-07-06 04:56:46,720 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: node01:50010:DataXceiver error processing READ_BLOCK operation src: /172.18.44.148:37180 dst: /172.18.44.145:50010
java.io.IOException: Not ready to serve the block pool, BP-515668640-172.18.44.143-1629272408048.
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:1354)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAccess(DataXceiver.java:1362)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:563)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:145)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:100)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:288)
at java.lang.Thread.run(Thread.java:748)
问题描述:
在namenode的50070端口,查看了一台Datanode有问题
修改hdfs-site.xml文件,dfs.datanode.max.transfer.threads的参数值把4096改为8192;
<property>
<name>dfs.datanode.max.transfer.threads</name>
<value>8192</value>
<description>
Specifies the maximum number of threads to use for transferring data
in and out of the DN.
</description>
</property>
先重启这台故障Datanode,注意重启时间有点长,重启完之后,再去把hdfs-site.xml替换其他的Datanode的hdfs-site.xml,需要一台重启成功之后,再重启其他的Datanode
其他:
1)jps可以查看datanode在不在;
2)之后会有个du进程,hadoop统计datanode存储数据使用情况,ps -ef | grep du可查看
3)查看datanode日志看到了gc,我的日志gc每隔一个小时报次,无影响;
4)ps-ef | grep datanode查看datanode的内存,-Xmx16384就是最大内存数16g