HDFS重启集群导致数据损坏，使用fsck命令修复过程

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰

版权声明：原创作品，谢绝转载！否则将追究法律责任。

 

 

 

 

　　我们先看一组输出

[root@flume112 ~]# hdfs fsck /user/kuaikan/report_new/2019/05/23/Connecting to namenode via http://calculation111.aggrx:50070FSCK started by root (auth:SIMPLE) from /10.1.2.112 for path /user/kuaikan/report_new/2019/05/23/ at Sat May 25 17:37:43 CST 2019....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Status: HEALTHY Total size:    48127214482 B (Total open files size: 72230205 B) Total dirs:    1 Total files:   5236 Total symlinks:                0 (Files currently being written: 26) Total blocks (validated):      5190 (avg. block size 9273066 B) (Total open file blocks (not validated): 26)　　　　#注意，这个26没有进行验证，说明有26个存在问题！ Minimally replicated blocks:   5190 (100.0 %) Over-replicated blocks:        0 (0.0 %) Under-replicated blocks:       0 (0.0 %) Mis-replicated blocks:         0 (0.0 %) Default replication factor:    3 Average block replication:     3.0 Corrupt blocks:                0 Missing replicas:              0 (0.0 %) Number of data-nodes:          18 Number of racks:               1FSCK ended at Sat May 25 17:37:43 CST 2019 in 163 millisecondsThe filesystem under path '/user/kuaikan/report_new/2019/05/23/' is HEALTHY[root@flume112 ~]#

 

　　官方关于fsck命令有对应的解释,我们查看更详细的信息，官方链接：。

[root@flume112 ~]# hdfs fsck /user/kuaikan/report_new/2019/05/23/  -openforwrite  Connecting to namenode via http://calculation111.aggrx:50070FSCK started by root (auth:SIMPLE) from /10.1.2.112 for path /user/kuaikan/report_new/2019/05/23/ at Sat May 25 17:40:23 CST 2019................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................/user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-45.10-19-64-160.1558554302405.txt 1163794 bytes, 1 block(s), OPENFORWRITE: ../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-45.10-9-109-248.1558554301696.txt 1209239 bytes, 1 block(s), OPENFORWRITE: ....../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-45.10-9-91-71.1558554301576.txt 1276007 bytes, 1 block(s), OPENFORWRITE: ........................................./user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-55.10-19-169-208.1558554901562.txt 1117716 bytes, 1 block(s), OPENFORWRITE: .../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-55.10-9-109-248.1558554901949.txt 1137697 bytes, 1 block(s), OPENFORWRITE: ...../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-55.10-9-88-233.1558554901647.txt 1096156 bytes, 1 block(s), OPENFORWRITE: ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-09-30.10-19-169-208.1558575001741.txt 8126635 bytes, 1 block(s), OPENFORWRITE: ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-12-55.10-19-147-93.1558587301545.txt 16778152 bytes, 1 block(s), OPENFORWRITE: ........../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-12-55.10-9-180-88.1558587301803.txt 16877467 bytes, 1 block(s), OPENFORWRITE: .........................../user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-13-01.10-9-180-88.1558587709627.txt 15320896 bytes, 1 block(s), OPENFORWRITE: ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................/user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-103-35.1558601101874.txt 314295 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-113-40.1558601102383.txt 363261 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-143-46.1558601101876.txt 1177059 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-145-58.1558601102255.txt 422652 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-147-93.1558601102225.txt 411212 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-157-38.1558601101874.txt 590968 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-169-208.1558601101557.txt 626874 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-64-160.1558601101782.txt 169801 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-95-33.1558601102283.txt 118028 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-19-98-150.1558601101668.txt 13544 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-109-248.1558601101427.txt 1112989 bytes, 1 block(s), OPENFORWRITE: ./user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-123-226.1558601101904.txt 1049811 bytes, 1 block(s), OPENFORWRITE: ./user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-176-96.1558601101582.txt 728802 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-180-88.1558601101302.txt 969764 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-88-233.1558601101531.txt 35615 bytes, 1 block(s), OPENFORWRITE: /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-16-45.10-9-91-71.1558601101720.txt 21771 bytes, 1 block(s), OPENFORWRITE: .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Status: HEALTHY Total size:    48199444687 B Total dirs:    1 Total files:   5262 Total symlinks:                0 Total blocks (validated):      5216 (avg. block size 9240691 B) Minimally replicated blocks:   5190 (99.50153 %) Over-replicated blocks:        0 (0.0 %) Under-replicated blocks:       0 (0.0 %) Mis-replicated blocks:         0 (0.0 %) Default replication factor:    3 Average block replication:     2.985046 Corrupt blocks:                0 Missing replicas:              0 (0.0 %) Number of data-nodes:          18 Number of racks:               1FSCK ended at Sat May 25 17:40:23 CST 2019 in 124 millisecondsThe filesystem under path '/user/kuaikan/report_new/2019/05/23/' is HEALTHY[root@flume112 ~]#

 

 

找到损坏的文件后，我们手动进行回复，注意，我们输入的必须是一个文件，并不是一个目录哟！

[root@calculation111 ~]# hdfs debug recoverLease -path /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-45.10-9-91-71.1558554301576.txt  -retries  3                                                    recoverLease returned false.Retrying in 5000 ms...Retry #1recoverLease SUCCEEDED on /user/kuaikan/report_new/2019/05/23/report.log.2019-05-23-03-45.10-9-91-71.1558554301576.txt[root@calculation111 ~]#