org.apache.hadoop.hive.ql.exec.spark.session

Class SparkSessionImpl

java.lang.Object

org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl

All Implemented Interfaces:

SparkSession

public class SparkSessionImpl
extends Object
implements SparkSession

Constructor Summary

Constructors

Constructor and Description
SparkSessionImpl()

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close session and release resources.
HiveConf	getConf()
ObjectPair<Long,Integer>	getMemoryAndCores() Get Spark shuffle memory per task, and total number of cores.
String	getSessionId()
boolean	isOpen()
static String	makeSessionId()
void	open(HiveConf conf) Initializes a Spark session for DAG execution.
SparkJobRef	submit(DriverContext driverContext, SparkWork sparkWork) Submit given sparkWork to SparkClient.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSessionImpl

public SparkSessionImpl()

Method Detail

open

public void open(HiveConf conf)
          throws HiveException

Description copied from interface: SparkSession

Initializes a Spark session for DAG execution.

Specified by:

open in interface SparkSession

Parameters:

conf - Hive configuration.

Throws:

HiveException

submit

public SparkJobRef submit(DriverContext driverContext,
                 SparkWork sparkWork)
                   throws Exception

Description copied from interface: SparkSession

Submit given sparkWork to SparkClient.

Specified by:

submit in interface SparkSession

Returns:

SparkJobRef

Throws:

Exception

getMemoryAndCores

public ObjectPair<Long,Integer> getMemoryAndCores()
                                           throws Exception

Description copied from interface: SparkSession

Get Spark shuffle memory per task, and total number of cores. This information can be used to estimate how many reducers a task can have.

Specified by:

getMemoryAndCores in interface SparkSession

Returns:

an object pair, the first element is the shuffle memory per task in bytes, the second element is the number of total cores usable by the client

Throws:

Exception

isOpen

public boolean isOpen()

Specified by:

isOpen in interface SparkSession

Returns:

true if the session is open and ready to submit jobs.

getConf

public HiveConf getConf()

Specified by:

getConf in interface SparkSession

Returns:

configuration.

getSessionId

public String getSessionId()

Specified by:

getSessionId in interface SparkSession

Returns:

session id.

close

public void close()

Description copied from interface: SparkSession

Close session and release resources.

Specified by:

close in interface SparkSession

makeSessionId

public static String makeSessionId()