| Package | Description |
|---|---|
| org.apache.pig |
Public interfaces and classes for Pig.
|
| org.apache.pig.backend.hadoop.executionengine | |
| org.apache.pig.backend.hadoop.executionengine.mapReduceLayer | |
| org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans | |
| org.apache.pig.backend.hadoop.executionengine.physicalLayer |
Implementation of physical operators that use hadoop as the execution engine
and data storage.
|
| org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators | |
| org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans | |
| org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators | |
| org.apache.pig.backend.hadoop.executionengine.physicalLayer.util | |
| org.apache.pig.backend.hadoop.executionengine.util | |
| org.apache.pig.impl.util | |
| org.apache.pig.newplan.logical.expression | |
| org.apache.pig.newplan.logical.relational | |
| org.apache.pig.pen |
| Modifier and Type | Method and Description |
|---|---|
protected PigStats |
PigServer.launchPlan(PhysicalPlan pp,
String jobName)
A common method for launching the jobs according to the physical plan
|
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
HExecutionEngine.compile(LogicalPlan plan,
Properties properties) |
| Modifier and Type | Field and Description |
|---|---|
PhysicalPlan |
MapReduceOper.combinePlan |
PhysicalPlan |
MapReduceOper.mapPlan |
protected PhysicalPlan |
PigGenericMapBase.mp |
PhysicalPlan |
MapReduceOper.reducePlan |
protected PhysicalPlan |
PigGenericMapReduce.Reduce.rp |
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
MRCompiler.getPlan()
Used to get the plan that was compiled
|
| Modifier and Type | Method and Description |
|---|---|
MROperPlan |
MapReduceLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
MapReduceLauncher.explain(PhysicalPlan php,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
abstract void |
Launcher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose)
Explain how a pig job will be executed on the underlying
infrastructure.
|
PigStats |
MapReduceLauncher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc) |
abstract PigStats |
Launcher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc)
Method to launch pig for hadoop either for a cluster's
job tracker or for a local job runner.
|
void |
PigGenericMapBase.setMapPlan(PhysicalPlan plan)
for local map/reduce simulation
|
void |
PigGenericMapReduce.Reduce.setReducePlan(PhysicalPlan plan)
Set the reduce plan: to be used by local runner for illustrator
|
| Constructor and Description |
|---|
MRCompiler(PhysicalPlan plan) |
MRCompiler(PhysicalPlan plan,
PigContext pigContext) |
PhyPlanSetter(PhysicalPlan plan) |
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
| Constructor and Description |
|---|
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
DotMRPrinter.InnerOperator.getPlan() |
| Modifier and Type | Method and Description |
|---|---|
void |
UDFFinder.setPlan(PhysicalPlan plan) |
| Constructor and Description |
|---|
DotMRPrinter.InnerOperator(PhysicalPlan plan,
String name) |
DotMRPrinter.InnerPlan(PhysicalPlan mapPlan,
PhysicalPlan combinePlan,
PhysicalPlan reducePlan) |
ScalarPhyFinder(PhysicalPlan plan) |
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
| Constructor and Description |
|---|
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
| Modifier and Type | Field and Description |
|---|---|
protected PhysicalPlan |
PhysicalOperator.parentPlan |
| Modifier and Type | Method and Description |
|---|---|
void |
PhysicalOperator.setParentPlan(PhysicalPlan physicalPlan) |
| Constructor and Description |
|---|
POPrinter(PrintStream ps,
PhysicalPlan pp) |
| Modifier and Type | Method and Description |
|---|---|
void |
UnaryExpressionOperator.setInputAsExpr(PhysicalPlan plan)
Set the contained expression to the be the input value.
|
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
PhysicalPlan.clone() |
| Modifier and Type | Method and Description |
|---|---|
protected Collection<PhysicalPlan> |
DotPOPrinter.getMultiOutputNestedPlans(PhysicalOperator op) |
protected Collection<PhysicalPlan> |
DotPOPrinter.getNestedPlans(PhysicalOperator op) |
| Modifier and Type | Method and Description |
|---|---|
protected DotPlanDumper |
DotPOPrinter.makeDumper(PhysicalPlan plan,
PrintStream ps) |
| Constructor and Description |
|---|
DotPOPrinter(PhysicalPlan plan,
PrintStream ps) |
DotPOPrinter(PhysicalPlan plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> subgraphs,
Set<Operator> multiInputSubgraphs,
Set<Operator> multiOutputSubgraphs) |
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
XMLPhysicalPlanPrinter(PhysicalPlan plan,
Document doc,
Element parent) |
| Constructor and Description |
|---|
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
| Modifier and Type | Field and Description |
|---|---|
protected List<PhysicalPlan> |
POForEach.inputPlans |
protected List<PhysicalPlan> |
POPreCombinerLocalRearrange.plans |
protected List<PhysicalPlan> |
POLocalRearrange.plans |
protected List<PhysicalPlan> |
POCollectedGroup.plans |
protected List<PhysicalPlan> |
POLocalRearrange.secondaryPlans |
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
POPartialAgg.getKeyPlan() |
PhysicalPlan |
POLimit.getLimitPlan() |
PhysicalPlan |
POFilter.getPlan() |
| Modifier and Type | Method and Description |
|---|---|
List<PhysicalPlan> |
POCounter.getCounterPlans() |
List<PhysicalPlan> |
POMergeJoin.getInnerPlansOf(int index) |
List<PhysicalPlan> |
POJoinPackage.getInputPlans() |
List<PhysicalPlan> |
POForEach.getInputPlans() |
MultiMap<PhysicalOperator,PhysicalPlan> |
POSkewedJoin.getJoinPlans() |
List<List<PhysicalPlan>> |
POFRJoin.getJoinPlans() |
List<PhysicalPlan> |
POMergeCogroup.getLRInnerPlansOf(int i) |
List<PhysicalPlan> |
POSplit.getPlans()
Returns the list of nested plans.
|
List<PhysicalPlan> |
POPreCombinerLocalRearrange.getPlans() |
List<PhysicalPlan> |
POLocalRearrange.getPlans() |
List<PhysicalPlan> |
PODemux.getPlans()
Returns the list of inner plans.
|
List<PhysicalPlan> |
POCollectedGroup.getPlans() |
List<PhysicalPlan> |
PORank.getRankPlans() |
List<PhysicalPlan> |
POSort.getSortPlans() |
List<PhysicalPlan> |
POPartialAgg.getValuePlans() |
| Modifier and Type | Method and Description |
|---|---|
void |
POForEach.addInputPlan(PhysicalPlan plan,
boolean flatten) |
void |
POSplit.addPlan(PhysicalPlan inPlan)
Appends the specified plan to the end of
the nested input plan list
|
void |
PODemux.addPlan(PhysicalPlan inPlan)
Appends the specified plan at the end of the list.
|
void |
POSplit.removePlan(PhysicalPlan plan)
Removes plan from
the nested input plan list
|
void |
POPartialAgg.setKeyPlan(PhysicalPlan keyPlan) |
void |
POLimit.setLimitPlan(PhysicalPlan expressionPlan) |
void |
POFilter.setPlan(PhysicalPlan plan) |
void |
POMergeJoin.setupRightPipeline(PhysicalPlan rightPipeline) |
| Modifier and Type | Method and Description |
|---|---|
void |
POCounter.setCounterPlans(List<PhysicalPlan> counterPlans) |
void |
POJoinPackage.setInputPlans(List<PhysicalPlan> plans) |
void |
POForEach.setInputPlans(List<PhysicalPlan> plans) |
void |
POSkewedJoin.setJoinPlans(MultiMap<PhysicalOperator,PhysicalPlan> joinPlans) |
void |
POPreCombinerLocalRearrange.setPlans(List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlans(List<PhysicalPlan> plans) |
void |
POCollectedGroup.setPlans(List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlansFromCombiner(List<PhysicalPlan> plans) |
void |
PORank.setRankPlans(List<PhysicalPlan> rankPlans) |
void |
POLocalRearrange.setSecondaryPlans(List<PhysicalPlan> plans) |
void |
POSort.setSortPlans(List<PhysicalPlan> sortPlans) |
void |
POPartialAgg.setValuePlans(List<PhysicalPlan> valuePlans) |
| Constructor and Description |
|---|
POCounter(OperatorKey operatorKey,
int requestedParallelism,
List inp,
List<PhysicalPlan> counterPlans,
List<Boolean> ascendingCol) |
POForEach(OperatorKey k,
int rp,
List<PhysicalPlan> inp,
List<Boolean> isToBeFlattened) |
POForEach(OperatorKey operatorKey,
int requestedParallelism,
List<PhysicalPlan> innerPlans,
List<Boolean> flattenList,
Schema schema) |
POFRJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
List<List<PhysicalPlan>> ppLists,
List<List<Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple) |
POFRJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
List<List<PhysicalPlan>> ppLists,
List<List<Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple,
Schema[] inputSchemas,
Schema[] keySchemas) |
POMergeJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
MultiMap<PhysicalOperator,PhysicalPlan> inpPlans,
List<List<Byte>> keyTypes,
LOJoin.JOINTYPE joinType,
Schema leftInputSchema,
Schema rightInputSchema,
Schema mergedInputSchema) |
POOptimizedForEach(OperatorKey k,
int rp,
List<PhysicalPlan> inp,
List<Boolean> isToBeFlattened) |
PORank(OperatorKey operatorKey,
int requestedParallelism,
List inp,
List<PhysicalPlan> rankPlans,
List<Boolean> ascendingCol) |
POSort(OperatorKey k,
int rp,
List inp,
List<PhysicalPlan> sortPlans,
List<Boolean> mAscCols,
POUserComparisonFunc mSortFunc) |
| Modifier and Type | Method and Description |
|---|---|
static <C extends PhysicalOperator> |
PlanHelper.containsPhysicalOperator(PhysicalPlan plan,
Class<C> opClass) |
static <C extends PhysicalOperator> |
PlanHelper.getPhysicalOperators(PhysicalPlan plan,
Class<C> opClass)
Returns a LinkedList of operators contained within the physical plan which implement the supplied class, in dependency order.
|
| Modifier and Type | Method and Description |
|---|---|
static FileSpec |
MapRedUtil.checkLeafIsStore(PhysicalPlan plan,
PigContext pigContext) |
| Modifier and Type | Method and Description |
|---|---|
static void |
CompilerUtils.addEmptyBagOuterJoin(PhysicalPlan fePlan,
Schema inputSchema) |
| Modifier and Type | Field and Description |
|---|---|
protected PhysicalPlan |
ExpToPhyTranslationVisitor.currentPlan |
| Modifier and Type | Field and Description |
|---|---|
protected Deque<PhysicalPlan> |
ExpToPhyTranslationVisitor.currentPlans |
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
ExpToPhyTranslationVisitor.getPhysicalPlan() |
| Constructor and Description |
|---|
ExpToPhyTranslationVisitor(OperatorPlan plan,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
PlanWalker walker,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
| Modifier and Type | Field and Description |
|---|---|
protected PhysicalPlan |
LogToPhyTranslationVisitor.currentPlan |
| Modifier and Type | Field and Description |
|---|---|
protected Deque<PhysicalPlan> |
LogToPhyTranslationVisitor.currentPlans |
| Modifier and Type | Method and Description |
|---|---|
PhysicalPlan |
LogToPhyTranslationVisitor.getPhysicalPlan() |
| Modifier and Type | Method and Description |
|---|---|
static void |
LogToPhyTranslationVisitor.updateWithEmptyBagCheck(PhysicalPlan fePlan,
Operator joinInput)
updates plan with check for empty bag and if bag is empty to flatten a bag
with as many null's as dictated by the schema
|
| Modifier and Type | Method and Description |
|---|---|
static Map<LogicalRelationalOperator,Collection<IdentityHashSet<Tuple>>> |
EquivalenceClasses.getLoToEqClassMap(PhysicalPlan plan,
LogicalPlan lp,
Map<Operator,PhysicalOperator> logToPhyMap,
Map<Operator,DataBag> logToDataMap,
Map<LOForEach,Map<LogicalRelationalOperator,PhysicalOperator>> forEachInnerLogToPhyMap,
HashMap<PhysicalOperator,Collection<IdentityHashSet<Tuple>>> poToEqclassesMap) |
void |
LocalMapReduceSimulator.launchPig(PhysicalPlan php,
Map<LOLoad,DataBag> baseData,
LineageTracer lineage,
IllustratorAttacher attacher,
ExampleGenerator eg,
PigContext pc) |
void |
IllustratorAttacher.revisit(PhysicalPlan plan)
revisit an enhanced physical plan from MR compilation
|
| Constructor and Description |
|---|
IllustratorAttacher(PhysicalPlan plan,
LineageTracer lineage,
int maxRecords,
Map<POLoad,LogicalSchema> poLoadToSchemaMap,
PigContext hadoopPigContext) |
LineageTrimmingVisitor(LogicalPlan plan,
Map<LOLoad,DataBag> baseData,
ExampleGenerator eg,
Map<Operator,PhysicalOperator> LogToPhyMap,
PhysicalPlan physPlan,
PigContext pc) |
PhysicalPlanResetter(PhysicalPlan plan) |
Copyright © 2007-2012 The Apache Software Foundation