org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POMultiQueryPackage

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMultiQueryPackage

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>, Illustrable

public class POMultiQueryPackage
extends POPackage

The package operator that packages the globally rearranged tuples into output format as required by multi-query de-multiplexer.

This operator is used when merging multiple Map-Reduce splittees into a Map-only splitter during multi-query optimization. The package operators of the reduce plans of the splittees form an indexed package list inside this operator. When this operator receives an input, it extracts the index from the key and calls the corresponding package to get the output data.

Due to the recursive nature of multi-query optimization, this operator may be contained in another multi-query packager.

The successor of this operator must be a PODemux operator which knows how to consume the output of this operator.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

POPackage.PackageType

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

distinct, isKeyCompound, isKeyTuple, keyAsTuple, keyInfo, mBagFactory, mTupleFactory

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, outputs, parentPlan, pigLogger, requestedParallelism, res, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POMultiQueryPackage(OperatorKey k) Constructs an operator with the specified key.
POMultiQueryPackage(OperatorKey k, int rp) Constructs an operator with the specified key and degree of parallelism.
POMultiQueryPackage(OperatorKey k, int rp, List<PhysicalOperator> inp) Constructs an operator with the specified key, degree of parallelism and inputs.
POMultiQueryPackage(OperatorKey k, List<PhysicalOperator> inp) Constructs an operator with the specified key and inputs.

Method Summary

Methods

Modifier and Type	Method and Description
void	addIsKeyWrappedList(List<Boolean> lst) Adds a list of IsKeyWrapped boolean values
void	addPackage(POPackage pack) Appends the specified package object to the end of the package list.
void	addPackage(POPackage pack, byte mapKeyType) Appends the specified package object to the end of the package list.
void	attachInput(PigNullableWritable k, Iterator<NullableTuple> inp) Attaches the required inputs
void	detachInput() attachInput's better half!
List<Boolean>	getIsKeyWrappedList() Returns the list of booleans that indicates if the key needs to unwrapped for the corresponding plan.
Result	getNextTuple() Constructs the output tuple from the inputs.
List<POPackage>	getPackages() Returns the list of packages.
boolean	isInCombiner()
boolean	isSameMapKeyType()
String	name()
void	setInCombiner(boolean inCombiner)
void	setSameMapKeyType(boolean sameMapKeyType)
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

clone, getInner, getKeyInfo, getKeyType, getNumInps, getPackageType, getValueTuple, illustratorMarkup, isDistinct, setDistinct, setInner, setKeyCompound, setKeyInfo, setKeyTuple, setKeyType, setNumInps, setPackageType, setUseSecondaryKey

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, cloneHelper, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k)

Constructs an operator with the specified key.

Parameters:

k - the operator key

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                   int rp)

Constructs an operator with the specified key and degree of parallelism.

Parameters:

k - the operator key

rp - the degree of parallelism requested

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                   List<PhysicalOperator> inp)

Constructs an operator with the specified key and inputs.

Parameters:

k - the operator key

inp - the inputs that this operator will read data from

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                   int rp,
                   List<PhysicalOperator> inp)

Constructs an operator with the specified key, degree of parallelism and inputs.

Parameters:

k - the operator key

rp - the degree of parallelism requested

inp - the inputs that this operator will read data from

Method Detail

name

public String name()

Overrides:

name in class POPackage

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Overrides:

supportsMultipleInputs in class POPackage

Returns:

true if it does, otherwise false.

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Overrides:

visit in class POPackage

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Overrides:

supportsMultipleOutputs in class POPackage

Returns:

true if it does, otherwise false.

attachInput

public void attachInput(PigNullableWritable k,
               Iterator<NullableTuple> inp)

Description copied from class: POPackage

Attaches the required inputs

Overrides:

attachInput in class POPackage

Parameters:

k - - the key being worked on

inp - - iterator of indexed tuples typically obtained from Hadoop

detachInput

public void detachInput()

Description copied from class: POPackage

attachInput's better half!

Overrides:

detachInput in class POPackage

addPackage

public void addPackage(POPackage pack)

Appends the specified package object to the end of the package list.

Parameters:

pack - package to be appended to the list

addPackage

public void addPackage(POPackage pack,
              byte mapKeyType)

Appends the specified package object to the end of the package list.

Parameters:

pack - package to be appended to the list

mapKeyType - the map key type associated with the package

getPackages

public List<POPackage> getPackages()

Returns the list of packages.

Returns:

the list of the packages

getNextTuple

public Result getNextTuple()
                    throws ExecException

Constructs the output tuple from the inputs.

The output is consumed by for the demultiplexer operator (PODemux) in the format (key, {bag of tuples}) where key is an indexed WritableComparable, not the wrapped value as a pig type.

Overrides:

getNextTuple in class POPackage

Throws:

ExecException

getIsKeyWrappedList

public List<Boolean> getIsKeyWrappedList()

Returns the list of booleans that indicates if the key needs to unwrapped for the corresponding plan.

Returns:

the list of isKeyWrapped boolean values

addIsKeyWrappedList

public void addIsKeyWrappedList(List<Boolean> lst)

Adds a list of IsKeyWrapped boolean values

Parameters:

lst - the list of boolean values to add

setInCombiner

public void setInCombiner(boolean inCombiner)

isInCombiner

public boolean isInCombiner()

setSameMapKeyType

public void setSameMapKeyType(boolean sameMapKeyType)

isSameMapKeyType

public boolean isSameMapKeyType()