org.apache.hadoop.hbase.mob.filecompactions

Class PartitionedMobFileCompactor

java.lang.Object

org.apache.hadoop.hbase.mob.filecompactions.MobFileCompactor

org.apache.hadoop.hbase.mob.filecompactions.PartitionedMobFileCompactor

@InterfaceAudience.Private
public class PartitionedMobFileCompactor
extends MobFileCompactor

An implementation of MobFileCompactor that compacts the mob files in partitions.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	compactionBatchSize The number of files compacted in a batch
protected int	compactionKVMax
protected int	delFileMaxCount
protected long	mergeableSize

Fields inherited from class org.apache.hadoop.hbase.mob.filecompactions.MobFileCompactor

column, conf, fs, mobFamilyDir, mobTableDir, pool, tableName

Constructor Summary

Constructors

Constructor and Description
PartitionedMobFileCompactor(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, TableName tableName, HColumnDescriptor column, ExecutorService pool)

Method Summary

Methods

Modifier and Type	Method and Description
List<org.apache.hadoop.fs.Path>	compact(List<org.apache.hadoop.fs.FileStatus> files, boolean isForceAllFiles) Compacts the candidate mob files.
protected List<org.apache.hadoop.fs.Path>	compactDelFiles(PartitionedMobFileCompactionRequest request, List<org.apache.hadoop.fs.Path> delFilePaths) Compacts the del files in batches which avoids opening too many files.
protected List<org.apache.hadoop.fs.Path>	compactMobFiles(PartitionedMobFileCompactionRequest request, List<StoreFile> delFiles) Compacts the selected small mob files and all the del files.
protected List<org.apache.hadoop.fs.Path>	performCompaction(PartitionedMobFileCompactionRequest request) Performs the compaction on the selected files.
protected PartitionedMobFileCompactionRequest	select(List<org.apache.hadoop.fs.FileStatus> candidates, boolean isForceAllFiles) Selects the compacted mob/del files.

Methods inherited from class org.apache.hadoop.hbase.mob.filecompactions.MobFileCompactor

compact, compact

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mergeableSize

protected long mergeableSize

delFileMaxCount

protected int delFileMaxCount

compactionBatchSize

protected int compactionBatchSize

The number of files compacted in a batch

compactionKVMax

protected int compactionKVMax

Constructor Detail

PartitionedMobFileCompactor

public PartitionedMobFileCompactor(org.apache.hadoop.conf.Configuration conf,
                           org.apache.hadoop.fs.FileSystem fs,
                           TableName tableName,
                           HColumnDescriptor column,
                           ExecutorService pool)

Method Detail

compact

public List<org.apache.hadoop.fs.Path> compact(List<org.apache.hadoop.fs.FileStatus> files,
                                      boolean isForceAllFiles)
                                        throws IOException

Description copied from class: MobFileCompactor

Compacts the candidate mob files.

Specified by:

compact in class MobFileCompactor

Parameters:

files - The candidate mob files.

isForceAllFiles - Whether add all mob files into the compaction.

Returns:

The paths of new mob files generated in the compaction.

Throws:

IOException

select

protected PartitionedMobFileCompactionRequest select(List<org.apache.hadoop.fs.FileStatus> candidates,
                                         boolean isForceAllFiles)
                                              throws IOException

Selects the compacted mob/del files. Iterates the candidates to find out all the del files and small mob files.

Parameters:

candidates - All the candidates.

isForceAllFiles - Whether add all mob files into the compaction.

Returns:

A compaction request.

Throws:

IOException

performCompaction

protected List<org.apache.hadoop.fs.Path> performCompaction(PartitionedMobFileCompactionRequest request)
                                                     throws IOException

Performs the compaction on the selected files.

1. Compacts the del files.
2. Compacts the selected small mob files and all the del files.
3. If all the candidates are selected, delete the del files.

Parameters:

request - The compaction request.

Returns:

The paths of new mob files generated in the compaction.

Throws:

IOException

compactMobFiles

protected List<org.apache.hadoop.fs.Path> compactMobFiles(PartitionedMobFileCompactionRequest request,
                                              List<StoreFile> delFiles)
                                                   throws IOException

Compacts the selected small mob files and all the del files.

Parameters:

request - The compaction request.

delFiles - The del files.

Returns:

The paths of new mob files after compactions.

Throws:

IOException

compactDelFiles

protected List<org.apache.hadoop.fs.Path> compactDelFiles(PartitionedMobFileCompactionRequest request,
                                              List<org.apache.hadoop.fs.Path> delFilePaths)
                                                   throws IOException

Compacts the del files in batches which avoids opening too many files.

Parameters:

request - The compaction request.

delFilePaths -

Returns:

The paths of new del files after merging or the original files if no merging is necessary.

Throws:

IOException