org.apache.hadoop.mapred

类 LocalJobRunner

java.lang.Object

org.apache.hadoop.mapred.LocalJobRunner

public class LocalJobRunner
extends Object

Implements MapReduce locally, in-process, for debugging.

字段概要

字段

限定符和类型	字段和说明
static String	LOCAL_MAX_MAPS
static org.apache.commons.logging.Log	LOG
static long	versionID

构造器概要

构造器

构造器和说明
LocalJobRunner(JobConf conf)

方法概要

方法

限定符和类型	方法和说明
void	cancelDelegationToken(Token<DelegationTokenIdentifier> token) Cancel a delegation token.
JobStatus[]	getAllJobs() Get all the jobs submitted.
TaskReport[]	getCleanupTaskReports(JobID id) Grab a bunch of info on the cleanup tasks that make up the job
ClusterStatus	getClusterStatus(boolean detailed) Get the current status of the cluster
Token<DelegationTokenIdentifier>	getDelegationToken(Text renewer) Get a new delegation token.
String	getFilesystemName() A MapReduce system always operates on a single filesystem.
Counters	getJobCounters(JobID id) Grab the current job counters
JobProfile	getJobProfile(JobID id) Grab a handle to a job that is already known to the JobTracker.
JobStatus[]	getJobsFromQueue(String queue) Gets all the jobs submitted to the particular Queue
JobStatus	getJobStatus(JobID id) Grab a handle to a job that is already known to the JobTracker.
static int	getLocalMaxRunningMaps(JobContext job)
TaskReport[]	getMapTaskReports(JobID id) Grab a bunch of info on the map tasks that make up the job
JobID	getNewJobId() Allocate a name for the job.
long	getProtocolVersion(String protocol, long clientVersion) Return protocol version corresponding to protocol interface.
QueueAclsInfo[]	getQueueAclsForCurrentUser() Gets the Queue ACLs for current user
AccessControlList	getQueueAdmins(String queueName) Get the administrators of the given job-queue.
JobQueueInfo	getQueueInfo(String queue) Gets scheduling information associated with the particular Job queue
JobQueueInfo[]	getQueues() Gets set of Job Queues associated with the Job Tracker
TaskReport[]	getReduceTaskReports(JobID id) Grab a bunch of info on the reduce tasks that make up the job
TaskReport[]	getSetupTaskReports(JobID id) Grab a bunch of info on the setup tasks that make up the job
String	getStagingAreaDir() Get a hint from the JobTracker where job-specific files are to be placed.
String	getSystemDir() Grab the jobtracker system directory path where job-specific files are to be placed.
TaskCompletionEvent[]	getTaskCompletionEvents(JobID jobid, int fromEventId, int maxEvents) Get task completion events for the jobid, starting from fromEventId.
String[]	getTaskDiagnostics(TaskAttemptID taskid) Returns the diagnostic information for a particular task in the given job.
JobStatus[]	jobsToComplete() Get the jobs that are not completed and not failed
void	killJob(JobID id) Kill the indicated job
boolean	killTask(TaskAttemptID taskId, boolean shouldFail) Throws UnsupportedOperationException
long	renewDelegationToken(Token<DelegationTokenIdentifier> token) Renew an existing delegation token
void	setJobPriority(JobID id, String jp) Set the priority of the specified job
static void	setLocalMaxRunningMaps(JobContext job, int maxMaps) Set the max number of map tasks to run concurrently in the LocalJobRunner.
JobStatus	submitJob(JobID jobid, String jobSubmitDir, Credentials credentials) Submit a Job for execution.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

public static final org.apache.commons.logging.Log LOG

LOCAL_MAX_MAPS

public static final String LOCAL_MAX_MAPS

另请参阅:

常量字段值

versionID

public static final long versionID

另请参阅:

常量字段值

构造器详细资料

LocalJobRunner

public LocalJobRunner(JobConf conf)
               throws IOException

抛出:

IOException

方法详细资料

getProtocolVersion

public long getProtocolVersion(String protocol,
                      long clientVersion)

从接口复制的说明: VersionedProtocol

Return protocol version corresponding to protocol interface.

参数:

protocol - The classname of the protocol interface

clientVersion - The version of the protocol that the client speaks

返回:

the version that the server will speak

getNewJobId

public JobID getNewJobId()

Allocate a name for the job.

返回:

a unique job name for submitting jobs.

submitJob

public JobStatus submitJob(JobID jobid,
                  String jobSubmitDir,
                  Credentials credentials)
                    throws IOException

Submit a Job for execution. Returns the latest profile for that job. The job files should be submitted in jobSubmitDir.

抛出:

IOException

killJob

public void killJob(JobID id)

Kill the indicated job

setJobPriority

public void setJobPriority(JobID id,
                  String jp)
                    throws IOException

Set the priority of the specified job

参数:

id - ID of the job

jp - Priority to be set for the job

抛出:

IOException

killTask

public boolean killTask(TaskAttemptID taskId,
               boolean shouldFail)
                 throws IOException

Throws UnsupportedOperationException

参数:

taskId - the id of the task to kill.

shouldFail - if true the task is failed and added to failed tasks list, otherwise it is just killed, w/o affecting job failure status.

抛出:

IOException

getJobProfile

public JobProfile getJobProfile(JobID id)

Grab a handle to a job that is already known to the JobTracker.

返回:

Profile of the job, or null if not found.

getMapTaskReports

public TaskReport[] getMapTaskReports(JobID id)

Grab a bunch of info on the map tasks that make up the job

getReduceTaskReports

public TaskReport[] getReduceTaskReports(JobID id)

Grab a bunch of info on the reduce tasks that make up the job

getCleanupTaskReports

public TaskReport[] getCleanupTaskReports(JobID id)

Grab a bunch of info on the cleanup tasks that make up the job

getSetupTaskReports

public TaskReport[] getSetupTaskReports(JobID id)

Grab a bunch of info on the setup tasks that make up the job

getJobStatus

public JobStatus getJobStatus(JobID id)

Grab a handle to a job that is already known to the JobTracker.

返回:

Status of the job, or null if not found.

getJobCounters

public Counters getJobCounters(JobID id)

Grab the current job counters

getFilesystemName

public String getFilesystemName()
                         throws IOException

A MapReduce system always operates on a single filesystem. This function returns the fs name. ('local' if the localfs; 'addr:port' if dfs). The client can then copy files into the right locations prior to submitting the job.

抛出:

IOException

getClusterStatus

public ClusterStatus getClusterStatus(boolean detailed)

Get the current status of the cluster

参数:

detailed - if true then report tracker names and memory usage

返回:

summary of the state of the cluster

jobsToComplete

public JobStatus[] jobsToComplete()

Get the jobs that are not completed and not failed

返回:

array of JobStatus for the running/to-be-run jobs.

getTaskCompletionEvents

public TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid,
                                            int fromEventId,
                                            int maxEvents)
                                              throws IOException

Get task completion events for the jobid, starting from fromEventId. Returns empty aray if no events are available.

参数:

jobid - job id

fromEventId - event id to start from.

maxEvents - the max number of events we want to look at

返回:

array of task completion events.

抛出:

IOException

getAllJobs

public JobStatus[] getAllJobs()

Get all the jobs submitted.

返回:

array of JobStatus for the submitted jobs

getTaskDiagnostics

public String[] getTaskDiagnostics(TaskAttemptID taskid)
                            throws IOException

Returns the diagnostic information for a particular task in the given job. To be implemented

参数:

taskid - the id of the task

返回:

an array of the diagnostic messages

抛出:

IOException

getSystemDir

public String getSystemDir()

Grab the jobtracker system directory path where job-specific files are to be placed.

返回:

the system directory where job-specific files are to be placed.

另请参阅:

JobSubmissionProtocol.getSystemDir()

getQueueAdmins

public AccessControlList getQueueAdmins(String queueName)
                                 throws IOException

Get the administrators of the given job-queue. This method is for hadoop internal use only.

返回:

Queue administrators ACL for the queue to which job is submitted to

抛出:

IOException

另请参阅:

org.apache.hadoop.mapred.JobSubmissionProtocol#getQueueAdmins()

getStagingAreaDir

public String getStagingAreaDir()
                         throws IOException

Get a hint from the JobTracker where job-specific files are to be placed.

返回:

the directory where job-specific files are to be placed.

抛出:

IOException

另请参阅:

JobSubmissionProtocol.getStagingAreaDir()

getJobsFromQueue

public JobStatus[] getJobsFromQueue(String queue)
                             throws IOException

Gets all the jobs submitted to the particular Queue

参数:

queue - Queue name

返回:

array of JobStatus for the submitted jobs

抛出:

IOException

getQueues

public JobQueueInfo[] getQueues()
                         throws IOException

Gets set of Job Queues associated with the Job Tracker

返回:

Array of the Job Queue Information Object

抛出:

IOException

getQueueInfo

public JobQueueInfo getQueueInfo(String queue)
                          throws IOException

Gets scheduling information associated with the particular Job queue

参数:

queue - Queue Name

返回:

Scheduling Information of the Queue

抛出:

IOException

getQueueAclsForCurrentUser

public QueueAclsInfo[] getQueueAclsForCurrentUser()
                                           throws IOException

Gets the Queue ACLs for current user

返回:

array of QueueAclsInfo object for current user.

抛出:

IOException

setLocalMaxRunningMaps

public static void setLocalMaxRunningMaps(JobContext job,
                          int maxMaps)

Set the max number of map tasks to run concurrently in the LocalJobRunner.

参数:

job - the job to configure

maxMaps - the maximum number of map tasks to allow.

getLocalMaxRunningMaps

public static int getLocalMaxRunningMaps(JobContext job)

返回:

the max number of map tasks to run concurrently in the LocalJobRunner.

cancelDelegationToken

public void cancelDelegationToken(Token<DelegationTokenIdentifier> token)
                           throws IOException,
                                  InterruptedException

Cancel a delegation token.

参数:

token - the token to cancel

抛出:

IOException

InterruptedException

getDelegationToken

public Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
                                                    throws IOException,
                                                           InterruptedException

Get a new delegation token.

参数:

renewer - the user other than the creator (if any) that can renew the token

返回:

the new delegation token

抛出:

IOException

InterruptedException

renewDelegationToken

public long renewDelegationToken(Token<DelegationTokenIdentifier> token)
                          throws IOException,
                                 InterruptedException

Renew an existing delegation token

参数:

token - the token to renew

返回:

the new expiration time

抛出:

IOException

InterruptedException