org.apache.hadoop.mapred.pipes

类 Submitter

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.mapred.pipes.Submitter

所有已实现的接口:

Configurable, Tool

public class Submitter
extends Configured
implements Tool

The main entry point and job submitter. It may either be used as a command line-based or API-based method to launch Pipes jobs.

字段概要

字段

限定符和类型	字段和说明
protected static org.apache.commons.logging.Log	LOG

构造器概要

构造器

构造器和说明
Submitter()
Submitter(Configuration conf)

方法概要

方法

限定符和类型	方法和说明
static String	getExecutable(JobConf conf) Get the URI of the application's executable.
static boolean	getIsJavaMapper(JobConf conf) Check whether the job is using a Java Mapper.
static boolean	getIsJavaRecordReader(JobConf conf) Check whether the job is using a Java RecordReader
static boolean	getIsJavaRecordWriter(JobConf conf) Will the reduce use a Java RecordWriter?
static boolean	getIsJavaReducer(JobConf conf) Check whether the job is using a Java Reducer.
static boolean	getKeepCommandFile(JobConf conf) Does the user want to keep the command file for debugging?
static RunningJob	jobSubmit(JobConf conf) Submit a job to the Map-Reduce framework.
static void	main(String[] args) Submit a pipes job based on the command line arguments.
int	run(String[] args) Execute the command with the given arguments.
static RunningJob	runJob(JobConf conf) Submit a job to the map/reduce cluster.
static void	setExecutable(JobConf conf, String executable) Set the URI for the application's executable.
static void	setIsJavaMapper(JobConf conf, boolean value) Set whether the Mapper is written in Java.
static void	setIsJavaRecordReader(JobConf conf, boolean value) Set whether the job is using a Java RecordReader.
static void	setIsJavaRecordWriter(JobConf conf, boolean value) Set whether the job will use a Java RecordWriter.
static void	setIsJavaReducer(JobConf conf, boolean value) Set whether the Reducer is written in Java.
static void	setKeepCommandFile(JobConf conf, boolean keep) Set whether to keep the command file for debugging
static RunningJob	submitJob(JobConf conf) 已过时。 Use runJob(JobConf)

从类继承的方法 org.apache.hadoop.conf.Configured

getConf, setConf

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.hadoop.conf.Configurable

getConf, setConf

字段详细资料

LOG

protected static final org.apache.commons.logging.Log LOG

构造器详细资料

Submitter

public Submitter()

Submitter

public Submitter(Configuration conf)

方法详细资料

getExecutable

public static String getExecutable(JobConf conf)

Get the URI of the application's executable.

参数:

conf -

返回:

the URI where the application's executable is located

setExecutable

public static void setExecutable(JobConf conf,
                 String executable)

Set the URI for the application's executable. Normally this is a hdfs: location.

参数:

conf -

executable - The URI of the application's executable.

setIsJavaRecordReader

public static void setIsJavaRecordReader(JobConf conf,
                         boolean value)

Set whether the job is using a Java RecordReader.

参数:

conf - the configuration to modify

value - the new value

getIsJavaRecordReader

public static boolean getIsJavaRecordReader(JobConf conf)

Check whether the job is using a Java RecordReader

参数:

conf - the configuration to check

返回:

is it a Java RecordReader?

setIsJavaMapper

public static void setIsJavaMapper(JobConf conf,
                   boolean value)

Set whether the Mapper is written in Java.

参数:

conf - the configuration to modify

value - the new value

getIsJavaMapper

public static boolean getIsJavaMapper(JobConf conf)

Check whether the job is using a Java Mapper.

参数:

conf - the configuration to check

返回:

is it a Java Mapper?

setIsJavaReducer

public static void setIsJavaReducer(JobConf conf,
                    boolean value)

Set whether the Reducer is written in Java.

参数:

conf - the configuration to modify

value - the new value

getIsJavaReducer

public static boolean getIsJavaReducer(JobConf conf)

Check whether the job is using a Java Reducer.

参数:

conf - the configuration to check

返回:

is it a Java Reducer?

setIsJavaRecordWriter

public static void setIsJavaRecordWriter(JobConf conf,
                         boolean value)

Set whether the job will use a Java RecordWriter.

参数:

conf - the configuration to modify

value - the new value to set

getIsJavaRecordWriter

public static boolean getIsJavaRecordWriter(JobConf conf)

Will the reduce use a Java RecordWriter?

参数:

conf - the configuration to check

返回:

true, if the output of the job will be written by Java

getKeepCommandFile

public static boolean getKeepCommandFile(JobConf conf)

Does the user want to keep the command file for debugging? If this is true, pipes will write a copy of the command data to a file in the task directory named "downlink.data", which may be used to run the C++ program under the debugger. You probably also want to set JobConf.setKeepFailedTaskFiles(true) to keep the entire directory from being deleted. To run using the data file, set the environment variable "hadoop.pipes.command.file" to point to the file.

参数:

conf - the configuration to check

返回:

will the framework save the command file?

setKeepCommandFile

public static void setKeepCommandFile(JobConf conf,
                      boolean keep)

Set whether to keep the command file for debugging

参数:

conf - the configuration to modify

keep - the new value

submitJob

@Deprecated
public static RunningJob submitJob(JobConf conf)
                            throws IOException

已过时。 Use runJob(JobConf)

Submit a job to the map/reduce cluster. All of the necessary modifications to the job to run under pipes are made to the configuration.

参数:

conf - the job to submit to the cluster (MODIFIED)

抛出:

IOException

runJob

public static RunningJob runJob(JobConf conf)
                         throws IOException

Submit a job to the map/reduce cluster. All of the necessary modifications to the job to run under pipes are made to the configuration.

参数:

conf - the job to submit to the cluster (MODIFIED)

抛出:

IOException

jobSubmit

public static RunningJob jobSubmit(JobConf conf)
                            throws IOException

Submit a job to the Map-Reduce framework. This returns a handle to the RunningJob which can be used to track the running-job.

参数:

conf - the job configuration.

返回:

a handle to the RunningJob which can be used to track the running-job.

抛出:

IOException

run

public int run(String[] args)
        throws Exception

从接口复制的说明: Tool

Execute the command with the given arguments.

指定者:

run 在接口中 Tool

参数:

args - command specific arguments.

返回:

exit code.

抛出:

Exception

main

public static void main(String[] args)
                 throws Exception

Submit a pipes job based on the command line arguments.

参数:

args -

抛出:

Exception