org.apache.hadoop.hbase.util.hbck

Class HFileCorruptionChecker

java.lang.Object

org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker

@InterfaceAudience.Private
public class HFileCorruptionChecker
extends Object

This class marches through all of the region's hfiles and verifies that they are all valid files. One just needs to instantiate the class, use checkTables(List<Path>) and then retrieve the corrupted hfiles (and quarantined files if in quarantining mode) The implementation currently parallelizes at the regionDir level.

Constructor Summary

Constructors

Constructor and Description
HFileCorruptionChecker(org.apache.hadoop.conf.Configuration conf, ExecutorService executor, boolean quarantine)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	checkColFamDir(org.apache.hadoop.fs.Path cfDir) Check all files in a column family dir.
protected void	checkHFile(org.apache.hadoop.fs.Path p) Checks a path to see if it is a valid hfile.
protected void	checkMobColFamDir(org.apache.hadoop.fs.Path cfDir) Check all files in a mob column family dir.
protected void	checkMobFile(org.apache.hadoop.fs.Path p) Checks a path to see if it is a valid mob file.
protected void	checkRegionDir(org.apache.hadoop.fs.Path regionDir) Check all column families in a region dir.
void	checkTables(Collection<org.apache.hadoop.fs.Path> tables) Check the specified table dirs for bad hfiles.
Collection<org.apache.hadoop.fs.Path>	getCorrupted()
Collection<org.apache.hadoop.fs.Path>	getCorruptedMobFiles()
Collection<org.apache.hadoop.fs.Path>	getFailureMobFiles()
Collection<org.apache.hadoop.fs.Path>	getFailures()
int	getHFilesChecked()
Collection<org.apache.hadoop.fs.Path>	getMissedMobFiles()
Collection<org.apache.hadoop.fs.Path>	getMissing()
int	getMobFilesChecked()
Collection<org.apache.hadoop.fs.Path>	getQuarantined()
Collection<org.apache.hadoop.fs.Path>	getQuarantinedMobFiles()
void	report(HBaseFsck.ErrorReporter out) Print a human readable summary of hfile quarantining operations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HFileCorruptionChecker

public HFileCorruptionChecker(org.apache.hadoop.conf.Configuration conf,
                      ExecutorService executor,
                      boolean quarantine)
                       throws IOException

Throws:

IOException

Method Detail

checkHFile

protected void checkHFile(org.apache.hadoop.fs.Path p)
                   throws IOException

Checks a path to see if it is a valid hfile.

Parameters:

p - full Path to an HFile

Throws:

IOException - This is a connectivity related exception

checkColFamDir

protected void checkColFamDir(org.apache.hadoop.fs.Path cfDir)
                       throws IOException

Check all files in a column family dir.

Parameters:

cfDir - column family directory

Throws:

IOException

checkMobColFamDir

protected void checkMobColFamDir(org.apache.hadoop.fs.Path cfDir)
                          throws IOException

Check all files in a mob column family dir.

Parameters:

cfDir - mob column family directory

Throws:

IOException

checkMobFile

protected void checkMobFile(org.apache.hadoop.fs.Path p)
                     throws IOException

Checks a path to see if it is a valid mob file.

Parameters:

p - full Path to a mob file.

Throws:

IOException - This is a connectivity related exception

checkRegionDir

protected void checkRegionDir(org.apache.hadoop.fs.Path regionDir)
                       throws IOException

Check all column families in a region dir.

Parameters:

regionDir - region directory

Throws:

IOException

checkTables

public void checkTables(Collection<org.apache.hadoop.fs.Path> tables)
                 throws IOException

Check the specified table dirs for bad hfiles.

Throws:

IOException

getFailures

public Collection<org.apache.hadoop.fs.Path> getFailures()

Returns:

the set of check failure file paths after checkTables is called.

getCorrupted

public Collection<org.apache.hadoop.fs.Path> getCorrupted()

Returns:

the set of corrupted file paths after checkTables is called.

getHFilesChecked

public int getHFilesChecked()

Returns:

number of hfiles checked in the last HfileCorruptionChecker run

getQuarantined

public Collection<org.apache.hadoop.fs.Path> getQuarantined()

Returns:

the set of successfully quarantined paths after checkTables is called.

getMissing

public Collection<org.apache.hadoop.fs.Path> getMissing()

Returns:

the set of paths that were missing. Likely due to deletion/moves from compaction or flushes.

getFailureMobFiles

public Collection<org.apache.hadoop.fs.Path> getFailureMobFiles()

Returns:

the set of check failure mob file paths after checkTables is called.

getCorruptedMobFiles

public Collection<org.apache.hadoop.fs.Path> getCorruptedMobFiles()

Returns:

the set of corrupted mob file paths after checkTables is called.

getMobFilesChecked

public int getMobFilesChecked()

Returns:

number of mob files checked in the last HfileCorruptionChecker run

getQuarantinedMobFiles

public Collection<org.apache.hadoop.fs.Path> getQuarantinedMobFiles()

Returns:

the set of successfully quarantined paths after checkTables is called.

getMissedMobFiles

public Collection<org.apache.hadoop.fs.Path> getMissedMobFiles()

Returns:

the set of paths that were missing. Likely due to table deletion or deletion/moves from compaction.

report

public void report(HBaseFsck.ErrorReporter out)

Print a human readable summary of hfile quarantining operations.

Parameters:

out -