org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class MapReducePOStoreImpl

java.lang.Object

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStoreImpl

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl

public class MapReducePOStoreImpl
extends POStoreImpl

This class is used to have a POStore write to DFS via a output collector/record writer. It sets up a modified job configuration to force a write to a specific subdirectory of the main output directory. This is done so that multiple output directories can be used in the same job.

Constructor Summary

Constructors

Constructor and Description
MapReducePOStoreImpl(org.apache.hadoop.mapreduce.TaskInputOutputContext context)

Method Summary

Methods

Modifier and Type	Method and Description
void	cleanUp() To perform cleanup when there is an error.
org.apache.hadoop.mapreduce.Counter	createRecordCounter(POStore store)
StoreFuncInterface	createStoreFunc(POStore store) Set up the storer
void	tearDown() At the end of processing, the outputstream is closed using this method

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapReducePOStoreImpl

public MapReducePOStoreImpl(org.apache.hadoop.mapreduce.TaskInputOutputContext context)

Method Detail

createStoreFunc

public StoreFuncInterface createStoreFunc(POStore store)
                                   throws IOException

Description copied from class: POStoreImpl

Set up the storer

Specified by:

createStoreFunc in class POStoreImpl

Parameters:

store - - the POStore object

Throws:

IOException

tearDown

public void tearDown()
              throws IOException

Description copied from class: POStoreImpl

At the end of processing, the outputstream is closed using this method

Overrides:

tearDown in class POStoreImpl

Throws:

IOException

cleanUp

public void cleanUp()
             throws IOException

Description copied from class: POStoreImpl

To perform cleanup when there is an error. Uses the FileLocalizer method which only deletes the file but not the dirs created with it.

Overrides:

cleanUp in class POStoreImpl

Throws:

IOException

createRecordCounter

public org.apache.hadoop.mapreduce.Counter createRecordCounter(POStore store)