org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POPackage

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>, Illustrable

Direct Known Subclasses:

POCombinerPackage, POJoinPackage, POMultiQueryPackage, POPackageLite

public class POPackage
extends PhysicalOperator

The package operator that packages the globally rearranged tuples into output format as required by co-group. This is last stage of processing co-group. This operator has a slightly different format than other operators in that, it takes two things as input. The key being worked on and the iterator of bags that contain indexed tuples that just need to be packaged into their appropriate output bags based on the index.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	POPackage.PackageType

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	distinct
protected boolean	isKeyCompound
protected boolean	isKeyTuple
protected Tuple	keyAsTuple
protected Map<Integer,Pair<Boolean,Map<Integer,Integer>>>	keyInfo
protected static BagFactory	mBagFactory
protected static TupleFactory	mTupleFactory

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, outputs, parentPlan, pigLogger, requestedParallelism, res, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POPackage(OperatorKey k)
POPackage(OperatorKey k, int rp)
POPackage(OperatorKey k, int rp, List<PhysicalOperator> inp)
POPackage(OperatorKey k, List<PhysicalOperator> inp)

Method Summary

Methods

Modifier and Type	Method and Description
void	attachInput(PigNullableWritable k, Iterator<NullableTuple> inp) Attaches the required inputs
POPackage	clone() Make a deep copy of this operator.
void	detachInput() attachInput's better half!
boolean[]	getInner()
Map<Integer,Pair<Boolean,Map<Integer,Integer>>>	getKeyInfo()
byte	getKeyType()
Result	getNextTuple() From the inputs, constructs the output tuple for this co-group in the required format which is (key, {bag of tuples from input 1}, {bag of tuples from input 2}, ...)
int	getNumInps()
POPackage.PackageType	getPackageType()
protected Tuple	getValueTuple(NullableTuple ntup, int index)
Tuple	illustratorMarkup(Object in, Object out, int eqClassIndex) input tuple mark up to be illustrate-able
boolean	isDistinct()
String	name()
void	setDistinct(boolean distinct)
void	setInner(boolean[] inner)
void	setKeyCompound(boolean keyCompound)
void	setKeyInfo(Map<Integer,Pair<Boolean,Map<Integer,Integer>>> keyInfo)
void	setKeyTuple(boolean keyTuple)
void	setKeyType(byte keyType)
void	setNumInps(int numInps)
void	setPackageType(POPackage.PackageType type)
void	setUseSecondaryKey(boolean useSecondaryKey)
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, cloneHelper, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

isKeyTuple

protected boolean isKeyTuple

isKeyCompound

protected boolean isKeyCompound

keyAsTuple

protected Tuple keyAsTuple

distinct

protected boolean distinct

keyInfo

protected Map<Integer,Pair<Boolean,Map<Integer,Integer>>> keyInfo

mBagFactory

protected static final BagFactory mBagFactory

mTupleFactory

protected static final TupleFactory mTupleFactory

Constructor Detail

POPackage

public POPackage(OperatorKey k)

POPackage

public POPackage(OperatorKey k,
         int rp)

POPackage

public POPackage(OperatorKey k,
         List<PhysicalOperator> inp)

POPackage

public POPackage(OperatorKey k,
         int rp,
         List<PhysicalOperator> inp)

Method Detail

name

public String name()

Specified by:

name in class Operator<PhyPlanVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class PhysicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

attachInput

public void attachInput(PigNullableWritable k,
               Iterator<NullableTuple> inp)

Attaches the required inputs

Parameters:

k - - the key being worked on

inp - - iterator of indexed tuples typically obtained from Hadoop

detachInput

public void detachInput()

attachInput's better half!

Overrides:

detachInput in class PhysicalOperator

getNumInps

public int getNumInps()

setNumInps

public void setNumInps(int numInps)

getInner

public boolean[] getInner()

setInner

public void setInner(boolean[] inner)

getNextTuple

public Result getNextTuple()
                    throws ExecException

From the inputs, constructs the output tuple for this co-group in the required format which is (key, {bag of tuples from input 1}, {bag of tuples from input 2}, ...)

Overrides:

getNextTuple in class PhysicalOperator

Throws:

ExecException

getValueTuple

protected Tuple getValueTuple(NullableTuple ntup,
                  int index)
                       throws ExecException

Throws:

ExecException

getKeyType

public byte getKeyType()

setKeyType

public void setKeyType(byte keyType)

clone

public POPackage clone()
                throws CloneNotSupportedException

Make a deep copy of this operator.

Overrides:

clone in class PhysicalOperator

Throws:

CloneNotSupportedException

See Also:

Do not use the clone method directly. Operators are cloned when logical plans are cloned using {@link LogicalPlanCloner}

setKeyInfo

public void setKeyInfo(Map<Integer,Pair<Boolean,Map<Integer,Integer>>> keyInfo)

Parameters:

keyInfo - the keyInfo to set

setKeyTuple

public void setKeyTuple(boolean keyTuple)

Parameters:

keyTuple - the keyTuple to set

setKeyCompound

public void setKeyCompound(boolean keyCompound)

Parameters:

keyCompound - the keyCompound to set

getKeyInfo

public Map<Integer,Pair<Boolean,Map<Integer,Integer>>> getKeyInfo()

Returns:

the keyInfo

isDistinct

public boolean isDistinct()

Returns:

the distinct

setDistinct

public void setDistinct(boolean distinct)

Parameters:

distinct - the distinct to set

setUseSecondaryKey

public void setUseSecondaryKey(boolean useSecondaryKey)

setPackageType

public void setPackageType(POPackage.PackageType type)

getPackageType

public POPackage.PackageType getPackageType()

illustratorMarkup

public Tuple illustratorMarkup(Object in,
                      Object out,
                      int eqClassIndex)

Description copied from interface: Illustrable

input tuple mark up to be illustrate-able

Parameters:

in - input tuple

out - output tuple before wrapped in ExampleTuple

eqClassIndex - index into equivalence classes in illustrator

Returns:

tuple