org.apache.hadoop.examples

Class AggregateWordCount.WordCountPlugInClass

java.lang.Object

org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor

org.apache.hadoop.examples.AggregateWordCount.WordCountPlugInClass

All Implemented Interfaces:

ValueAggregatorDescriptor

Enclosing class:

AggregateWordCount

public static class AggregateWordCount.WordCountPlugInClass
extends ValueAggregatorBaseDescriptor

Field Summary

Fields inherited from class org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor

DOUBLE_VALUE_SUM, inputFile, LONG_VALUE_MAX, LONG_VALUE_MIN, LONG_VALUE_SUM, STRING_VALUE_MAX, STRING_VALUE_MIN, UNIQ_VALUE_COUNT, VALUE_HISTOGRAM

Fields inherited from interface org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorDescriptor

ONE, TYPE_SEPARATOR

Constructor Summary

Constructors

Constructor and Description
AggregateWordCount.WordCountPlugInClass()

Method Summary

Methods

Modifier and Type	Method and Description
ArrayList<Map.Entry<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>>	generateKeyValPairs(Object key, Object val) Generate 1 or 2 aggregation-id/value pairs for the given key/value pair.

Methods inherited from class org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor

configure, generateEntry, generateValueAggregator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AggregateWordCount.WordCountPlugInClass

public AggregateWordCount.WordCountPlugInClass()

Method Detail

generateKeyValPairs

public ArrayList<Map.Entry<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>> generateKeyValPairs(Object key,
                                                                                            Object val)

Description copied from class: ValueAggregatorBaseDescriptor

Generate 1 or 2 aggregation-id/value pairs for the given key/value pair. The first id will be of type LONG_VALUE_SUM, with "record_count" as its aggregation id. If the input is a file split, the second id of the same type will be generated too, with the file name as its aggregation id. This achieves the behavior of counting the total number of records in the input data, and the number of records in each input file.

Specified by:

generateKeyValPairs in interface ValueAggregatorDescriptor

Overrides:

generateKeyValPairs in class ValueAggregatorBaseDescriptor

Parameters:

key - input key

val - input value

Returns:

a list of aggregation id/value pairs. An aggregation id encodes an aggregation type which is used to guide the way to aggregate the value in the reduce/combiner phrase of an Aggregate based job.