org.apache.hadoop.hive.ql.exec

Class DummyStoreOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<DummyStoreDesc>

org.apache.hadoop.hive.ql.exec.DummyStoreOperator

All Implemented Interfaces:

Serializable, Cloneable, Node

Direct Known Subclasses:

TezDummyStoreOperator

public class DummyStoreOperator
extends Operator<DummyStoreDesc>
implements Serializable

For SortMerge joins, this is a dummy operator, which stores the row for the small table before it reaches the sort merge join operator. Consider a query like: select * from (subq1 --> has a filter) join (subq2 --> has a filter) on some key Let us assume that subq1 is the small table (either specified by the user or inferred automatically). Since there can be multiple buckets/partitions for the table corresponding to subq1 given a file in subq2, a priority queue is present in SMBMapJoinOperator to scan the various buckets and fetch the least row (corresponding to the join key). The tree corresponding to subq1 needs to be evaluated in order to compute the join key (since the select list for the join key can move across different object inspectors). Therefore the following operator tree is created: TableScan (subq1) --> Select --> Filter --> DummyStore \ \ SMBJoin / / TableScan (subq2) --> Select --> Filter In order to fetch the row with the least join key from the small table, the row from subq1 is partially processed, and stored in DummyStore. For the actual processing of the join, SMBJoin (child of DummyStore) is processed for the transformed row. Note that in the absence of support for joins for sub-queries, this was not needed, since all transformations were done after SMBJoin, or for the small tables, nothing could have been present between TableScan and SMBJoin.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected InspectableObject	result

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

alias, childOperators, childOperatorsArray, childOperatorsTag, colExprMap, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVECOUNTERCREATEDFILES, HIVECOUNTERFATAL, id, inputObjInspectors, isLogDebugEnabled, isLogInfoEnabled, isLogTraceEnabled, LOG, operatorId, out, outputObjInspector, parentOperators, PLOG, reporter, state, statsMap

Constructor Summary

Constructors

Constructor and Description
DummyStoreOperator()

Method Summary

Methods

Modifier and Type	Method and Description
InspectableObject	getResult()
OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
protected Collection<Future<?>>	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
void	process(Object row, int tag) Process the row.
void	reset()

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, closeOp, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, endGroup, flush, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getConf, getConfiguration, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getName, getNextCntr, getNumChild, getNumParent, getOperatorId, getOperatorName, getOpTraits, getOutputObjInspector, getParentOperators, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, processGroup, removeChild, removeChildAndAdoptItsChildren, removeChildren, removeParent, replaceChild, replaceParent, resetId, resetStats, setAlias, setChildOperators, setColumnExprMap, setConf, setDone, setExecContext, setGroupKeyObject, setId, setInputContext, setInputObjInspectors, setOperatorId, setOpTraits, setOutputCollector, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

result

protected transient InspectableObject result

Constructor Detail

DummyStoreOperator

public DummyStoreOperator()

Method Detail

initializeOp

protected Collection<Future<?>> initializeOp(org.apache.hadoop.conf.Configuration hconf)
                                      throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<DummyStoreDesc>

Throws:

HiveException

process

public void process(Object row,
           int tag)
             throws HiveException

Description copied from class: Operator

Process the row.

Specified by:

process in class Operator<DummyStoreDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException

reset

public void reset()

Overrides:

reset in class Operator<DummyStoreDesc>

getResult

public InspectableObject getResult()

getType

public OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<DummyStoreDesc>

Returns:

OperatorType.*