org.apache.hadoop.hive.ql.exec.spark.session

Interface SparkSession

All Known Implementing Classes:

SparkSessionImpl

public interface SparkSession

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close session and release resources.
HiveConf	getConf()
ObjectPair<Long,Integer>	getMemoryAndCores() Get Spark shuffle memory per task, and total number of cores.
String	getSessionId()
boolean	isOpen()
void	open(HiveConf conf) Initializes a Spark session for DAG execution.
SparkJobRef	submit(DriverContext driverContext, SparkWork sparkWork) Submit given sparkWork to SparkClient.

Method Detail

open

void open(HiveConf conf)
          throws HiveException

Initializes a Spark session for DAG execution.

Parameters:

conf - Hive configuration.

Throws:

HiveException

submit

SparkJobRef submit(DriverContext driverContext,
                 SparkWork sparkWork)
                   throws Exception

Submit given sparkWork to SparkClient.

Parameters:

driverContext -

sparkWork -

Returns:

SparkJobRef

Throws:

Exception

getMemoryAndCores

ObjectPair<Long,Integer> getMemoryAndCores()
                                           throws Exception

Get Spark shuffle memory per task, and total number of cores. This information can be used to estimate how many reducers a task can have.

Returns:

an object pair, the first element is the shuffle memory per task in bytes, the second element is the number of total cores usable by the client

Throws:

Exception

isOpen

boolean isOpen()

Returns:

true if the session is open and ready to submit jobs.

getConf

HiveConf getConf()

Returns:

configuration.

getSessionId

String getSessionId()

Returns:

session id.

close

void close()

Close session and release resources.