org.apache.hadoop.hive.ql.optimizer.spark

Class SparkSortMergeJoinOptimizer

java.lang.Object

org.apache.hadoop.hive.ql.optimizer.AbstractBucketJoinProc

org.apache.hadoop.hive.ql.optimizer.AbstractSMBJoinProc

org.apache.hadoop.hive.ql.optimizer.spark.SparkSortMergeJoinOptimizer

All Implemented Interfaces:

NodeProcessor

public class SparkSortMergeJoinOptimizer
extends AbstractSMBJoinProc
implements NodeProcessor

Converts a common join operator to an SMB join if eligible. Handles auto SMB conversion.

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.optimizer.AbstractBucketJoinProc

pGraphContext

Constructor Summary

Constructors

Constructor and Description
SparkSortMergeJoinOptimizer()
SparkSortMergeJoinOptimizer(ParseContext pctx)

Method Summary

Methods

Modifier and Type	Method and Description
protected boolean	canConvertJoinToSMBJoin(JoinOperator joinOperator, SortBucketJoinProcCtx smbJoinContext, ParseContext pGraphContext, Stack<Node> stack)
protected SMBMapJoinOperator	convertJoinToSMBJoinAndReturn(JoinOperator joinOp, SortBucketJoinProcCtx smbJoinContext)
Object	process(Node nd, Stack<Node> stack, NodeProcessorCtx procCtx, Object... nodeOutputs) Generic process for all ops that don't have specific implementations.

Methods inherited from class org.apache.hadoop.hive.ql.optimizer.AbstractSMBJoinProc

canConvertBucketMapJoinToSMBJoin, canConvertJoinToBucketMapJoin, canConvertJoinToSMBJoin, checkConvertJoinToSMBJoin, convertBucketMapJoinToSMBJoin, convertJoinToBucketMapJoin, convertJoinToSMBJoin

Methods inherited from class org.apache.hadoop.hive.ql.optimizer.AbstractBucketJoinProc

canConvertMapJoinToBucketMapJoin, checkConvertBucketMapJoin, convertMapJoinToBucketMapJoin, getBucketFilePathsOfPartition, toColumns

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSortMergeJoinOptimizer

public SparkSortMergeJoinOptimizer(ParseContext pctx)

SparkSortMergeJoinOptimizer

public SparkSortMergeJoinOptimizer()

Method Detail

process

public Object process(Node nd,
             Stack<Node> stack,
             NodeProcessorCtx procCtx,
             Object... nodeOutputs)
               throws SemanticException

Description copied from interface: NodeProcessor

Generic process for all ops that don't have specific implementations.

Specified by:

process in interface NodeProcessor

Specified by:

process in class AbstractSMBJoinProc

Parameters:

nd - operator to process

procCtx - operator processor context

nodeOutputs - A variable argument list of outputs from other nodes in the walk

Returns:

Object to be returned by the process call

Throws:

SemanticException

canConvertJoinToSMBJoin

protected boolean canConvertJoinToSMBJoin(JoinOperator joinOperator,
                              SortBucketJoinProcCtx smbJoinContext,
                              ParseContext pGraphContext,
                              Stack<Node> stack)
                                   throws SemanticException

Throws:

SemanticException

convertJoinToSMBJoinAndReturn

protected SMBMapJoinOperator convertJoinToSMBJoinAndReturn(JoinOperator joinOp,
                                               SortBucketJoinProcCtx smbJoinContext)
                                                    throws SemanticException

Throws:

SemanticException