org.apache.hadoop.hbase.mob

Class DefaultMobCompactor

java.lang.Object

org.apache.hadoop.hbase.regionserver.compactions.Compactor

org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor

org.apache.hadoop.hbase.mob.DefaultMobCompactor

@InterfaceAudience.Private
public class DefaultMobCompactor
extends DefaultCompactor

Compact passed set of files in the mob-enabled column family.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.regionserver.compactions.Compactor

Compactor.CellSink, Compactor.FileDetails

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.compactions.Compactor

compactionCompression, compactionKVMax, conf, keepSeqIdPeriod, progress, store

Constructor Summary

Constructors

Constructor and Description
DefaultMobCompactor(org.apache.hadoop.conf.Configuration conf, Store store)

Method Summary

Methods

Modifier and Type	Method and Description
protected InternalScanner	createScanner(Store store, List<StoreFileScanner> scanners, ScanType scanType, long smallestReadPoint, long earliestPutTs)
protected StoreFile.Writer	createTmpWriter(Compactor.FileDetails fd, boolean shouldDropBehind) Creates a writer for a new file in a temporary directory.
protected boolean	performCompaction(Compactor.FileDetails fd, InternalScanner scanner, Compactor.CellSink writer, long smallestReadPoint, boolean cleanSeqId, CompactionThroughputController throughputController, boolean major) Performs compaction on a column family with the mob flag enabled.

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor

compact, compactForTesting

Methods inherited from class org.apache.hadoop.hbase.regionserver.compactions.Compactor

appendMetadataAndCloseWriter, createFileScanners, createScanner, getFileDetails, getProgress, getSmallestReadPoint, postCreateCoprocScanner, preCreateCoprocScanner, preCreateCoprocScanner, resetSeqId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultMobCompactor

public DefaultMobCompactor(org.apache.hadoop.conf.Configuration conf,
                   Store store)

Method Detail

createTmpWriter

protected StoreFile.Writer createTmpWriter(Compactor.FileDetails fd,
                               boolean shouldDropBehind)
                                    throws IOException

Creates a writer for a new file in a temporary directory.

Overrides:

createTmpWriter in class DefaultCompactor

Parameters:

fd - The file details.

shouldDropBehind - Should the writer drop behind.

Returns:

Writer for a new StoreFile in the tmp dir.

Throws:

IOException

createScanner

protected InternalScanner createScanner(Store store,
                            List<StoreFileScanner> scanners,
                            ScanType scanType,
                            long smallestReadPoint,
                            long earliestPutTs)
                                 throws IOException

Overrides:

createScanner in class Compactor

Parameters:

store - store

scanners - Store file scanners.

scanType - Scan type.

smallestReadPoint - Smallest MVCC read point.

earliestPutTs - Earliest put across all files.

Returns:

A compaction scanner.

Throws:

IOException

performCompaction

protected boolean performCompaction(Compactor.FileDetails fd,
                        InternalScanner scanner,
                        Compactor.CellSink writer,
                        long smallestReadPoint,
                        boolean cleanSeqId,
                        CompactionThroughputController throughputController,
                        boolean major)
                             throws IOException

Performs compaction on a column family with the mob flag enabled. This is for when the mob threshold size has changed or if the mob column family mode has been toggled via an alter table statement. Compacts the files by the following rules. 1. If the cell has a mob reference tag, the cell's value is the path of the mob file.

1. If the value size of a cell is larger than the threshold, this cell is regarded as a mob, directly copy the (with mob tag) cell into the new store file.
2. Otherwise, retrieve the mob cell from the mob file, and writes a copy of the cell into the new store file.

2. If the cell doesn't have a reference tag.

1. If the value size of a cell is larger than the threshold, this cell is regarded as a mob, write this cell to a mob file, and write the path of this mob file to the store file.
2. Otherwise, directly write this cell into the store file.

In the mob compaction, the MobCompactionStoreScanner is used as a scanner which could output the normal cells and delete markers together when required. After the major compaction on the normal hfiles, we have a guarantee that we have purged all deleted or old version mob refs, and the delete markers are written to a del file with the suffix _del. Because of this, it is safe to use the del file in the mob compaction. The mob compaction doesn't take place in the normal hfiles, it occurs directly in the mob files. When the small mob files are merged into bigger ones, the del file is added into the scanner to filter the deleted cells.

Overrides:

performCompaction in class Compactor

Parameters:

fd - File details

scanner - Where to read from.

writer - Where to write to.

smallestReadPoint - Smallest read point.

cleanSeqId - When true, remove seqId(used to be mvcc) value which is <= smallestReadPoint

major - Is a major compaction.

Returns:

Whether compaction ended; false if it was interrupted for any reason.

Throws:

IOException