org.apache.hadoop.hbase.mob.mapreduce

Class SweepReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,KeyValue,org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>

org.apache.hadoop.hbase.mob.mapreduce.SweepReducer

@InterfaceAudience.Private
public class SweepReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,KeyValue,org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>

The reducer of a sweep job. This reducer merges the small mob files into bigger ones, and write visited names of mob files to a sequence file which is used by the sweep job to delete the unused mob files. The key of the input is a file name, the value is a collection of KeyValue where the KeyValue is the actual cell (its format is valueLength + fileName) in HBase. In this reducer, we could know how many cells exist in HBase for a mob file. If the existCellSize/mobFileSize < compactionRatio, this mob file needs to be merged.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	SweepReducer.SweepPartition The mob files which have the same start key and date are in the same partition.
static class	SweepReducer.SweepPartitionId The sweep partition id.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Constructor Summary

Constructors

Constructor and Description
SweepReducer()

Method Summary

Methods

Modifier and Type	Method and Description
void	run(org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

cleanup, reduce

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SweepReducer

public SweepReducer()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,KeyValue,org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>

Throws:

IOException

InterruptedException

run

public void run(org.apache.hadoop.mapreduce.Reducer.Context context)
         throws IOException,
                InterruptedException

Overrides:

run in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,KeyValue,org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>

Throws:

IOException

InterruptedException