程序包 | 说明 |
---|---|
org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
限定符和类型 | 方法和说明 |
---|---|
static TaskAttemptID |
TaskAttemptID.downgrade(TaskAttemptID old)
Downgrade a new TaskAttemptID to an old one
|
static TaskAttemptID |
TaskAttemptID.forName(String str)
Construct a TaskAttemptID object from given string
|
TaskAttemptID |
TaskReport.getSuccessfulTaskAttempt()
Get the attempt ID that took this task to completion
|
TaskAttemptID |
TaskCompletionEvent.getTaskAttemptId()
Returns task id.
|
TaskAttemptID |
TaskAttemptContext.getTaskAttemptID()
Get the taskAttemptID.
|
TaskAttemptID |
TaskStatus.getTaskID() |
TaskAttemptID |
Task.getTaskID() |
static TaskAttemptID |
TaskAttemptID.read(DataInput in)
已过时。
|
限定符和类型 | 方法和说明 |
---|---|
List<TaskAttemptID> |
TaskStatus.getFetchFailedMaps()
Get the list of maps from which output-fetches failed.
|
Collection<TaskAttemptID> |
TaskReport.getRunningTaskAttempts()
Get the running task attempt IDs for this task
|
限定符和类型 | 方法和说明 |
---|---|
boolean |
TaskUmbilicalProtocol.canCommit(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
Polling to know whether the task can go-ahead with commit
|
boolean |
TaskTracker.canCommit(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
Child checking whether it can commit
|
void |
TaskUmbilicalProtocol.commitPending(TaskAttemptID taskId,
TaskStatus taskStatus,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report that the task is complete, but its commit is pending.
|
void |
TaskTracker.commitPending(TaskAttemptID taskid,
TaskStatus taskStatus,
org.apache.hadoop.mapred.JvmContext jvmContext)
Task is reporting that it is in commit_pending
and it is waiting for the commit Response
|
void |
TaskTrackerMetricsSource.completeTask(TaskAttemptID t) |
abstract void |
TaskController.createLogDir(TaskAttemptID taskID,
boolean isCleanup)
Creates task log dir
|
void |
DefaultTaskController.createLogDir(TaskAttemptID taskID,
boolean isCleanup) |
static void |
TaskLog.createTaskAttemptLogDir(TaskAttemptID taskID,
boolean isCleanup,
String[] localDirs)
Create log directory for the given attempt.
|
void |
TaskUmbilicalProtocol.done(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report that the task is successfully completed.
|
void |
TaskTracker.done(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
The task is done.
|
void |
JobInProgress.failedTask(org.apache.hadoop.mapred.TaskInProgress tip,
TaskAttemptID taskid,
String reason,
TaskStatus.Phase phase,
TaskStatus.State state,
String trackerName)
Fail a task with a given reason, but without a status object.
|
void |
TaskUmbilicalProtocol.fatalError(TaskAttemptID taskId,
String message,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report that the task encounted a fatal error.
|
void |
TaskTracker.fatalError(TaskAttemptID taskId,
String msg,
org.apache.hadoop.mapred.JvmContext jvmContext)
A child task had a fatal error.
|
void |
TaskUmbilicalProtocol.fsError(TaskAttemptID taskId,
String message,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report that the task encounted a local filesystem error.
|
void |
TaskTracker.fsError(TaskAttemptID taskId,
String message,
org.apache.hadoop.mapred.JvmContext jvmContext)
A child task had a local filesystem error.
|
String |
JobTracker.getAssignedTracker(TaskAttemptID taskId)
Get tracker name for a given task id.
|
MapTaskCompletionEventsUpdate |
TaskUmbilicalProtocol.getMapCompletionEvents(JobID jobId,
int fromIndex,
int maxLocs,
TaskAttemptID id,
org.apache.hadoop.mapred.JvmContext jvmContext)
Called by a reduce task to get the map output locations for finished maps.
|
MapTaskCompletionEventsUpdate |
TaskTracker.getMapCompletionEvents(JobID jobId,
int fromEventId,
int maxLocs,
TaskAttemptID id,
org.apache.hadoop.mapred.JvmContext jvmContext) |
static String |
TaskLog.getTaskAttemptLogDir(TaskAttemptID taskID,
String cleanupSuffix,
String[] localDirs)
Get attempt log directory path for the given attempt-id under randomly
selected mapred local directory.
|
String[] |
RunningJob.getTaskDiagnostics(TaskAttemptID taskid)
Gets the diagnostic messages for a given task attempt.
|
String[] |
LocalJobRunner.getTaskDiagnostics(TaskAttemptID taskid)
Returns the diagnostic information for a particular task in the given job.
|
String[] |
JobTracker.getTaskDiagnostics(TaskAttemptID taskId)
Get the diagnostics for a given task
|
static File |
TaskLog.getTaskLogFile(TaskAttemptID taskid,
boolean isCleanup,
TaskLog.LogName filter) |
void |
RunningJob.killTask(TaskAttemptID taskId,
boolean shouldFail)
Kill indicated task attempt.
|
boolean |
LocalJobRunner.killTask(TaskAttemptID taskId,
boolean shouldFail)
|
boolean |
JobTracker.killTask(TaskAttemptID taskid,
boolean shouldFail) |
static void |
JobHistory.MapAttempt.logFailed(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error)
|
static void |
JobHistory.ReduceAttempt.logFailed(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error)
|
static void |
JobHistory.MapAttempt.logFailed(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error,
String taskType)
Log task attempt failed event.
|
static void |
JobHistory.ReduceAttempt.logFailed(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error,
String taskType)
Log failed reduce task attempt.
|
static void |
JobHistory.Task.logFailed(TaskID taskId,
String taskType,
long time,
String error,
TaskAttemptID failedDueToAttempt) |
static void |
JobHistory.ReduceAttempt.logFinished(TaskAttemptID taskAttemptId,
long shuffleFinished,
long sortFinished,
long finishTime,
String hostName)
|
static void |
JobHistory.ReduceAttempt.logFinished(TaskAttemptID taskAttemptId,
long shuffleFinished,
long sortFinished,
long finishTime,
String hostName,
String taskType,
String stateString,
Counters counter)
Log finished event of this task.
|
static void |
JobHistory.MapAttempt.logFinished(TaskAttemptID taskAttemptId,
long finishTime,
String hostName)
|
static void |
JobHistory.MapAttempt.logFinished(TaskAttemptID taskAttemptId,
long finishTime,
String hostName,
String taskType,
String stateString,
Counters counter)
Log finish time of map task attempt.
|
static void |
JobHistory.MapAttempt.logKilled(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error)
|
static void |
JobHistory.ReduceAttempt.logKilled(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error)
|
static void |
JobHistory.MapAttempt.logKilled(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error,
String taskType)
Log task attempt killed event.
|
static void |
JobHistory.ReduceAttempt.logKilled(TaskAttemptID taskAttemptId,
long timestamp,
String hostName,
String error,
String taskType)
Log killed reduce task attempt.
|
static void |
JobHistory.MapAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String hostName)
|
static void |
JobHistory.ReduceAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String hostName)
|
static void |
JobHistory.MapAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String trackerName,
int httpPort,
String taskType)
已过时。
|
static void |
JobHistory.ReduceAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String trackerName,
int httpPort,
String taskType)
已过时。
|
static void |
JobHistory.MapAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String trackerName,
int httpPort,
String taskType,
org.apache.hadoop.mapred.Locality locality,
org.apache.hadoop.mapred.Avataar avataar)
Log start time of this map task attempt.
|
static void |
JobHistory.ReduceAttempt.logStarted(TaskAttemptID taskAttemptId,
long startTime,
String trackerName,
int httpPort,
String taskType,
org.apache.hadoop.mapred.Locality locality,
org.apache.hadoop.mapred.Avataar avataar)
Log start time of Reduce task attempt.
|
void |
TaskTracker.mapOutputLost(TaskAttemptID taskid,
String errorMsg)
A completed map task's output has been lost.
|
boolean |
TaskUmbilicalProtocol.ping(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
Periodically called by child to check if parent is still alive.
|
boolean |
TaskTracker.ping(TaskAttemptID taskid,
org.apache.hadoop.mapred.JvmContext jvmContext)
Child checking to see if we're alive.
|
void |
TaskUmbilicalProtocol.reportDiagnosticInfo(TaskAttemptID taskid,
String trace,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report error messages back to parent.
|
void |
TaskTracker.reportDiagnosticInfo(TaskAttemptID taskid,
String info,
org.apache.hadoop.mapred.JvmContext jvmContext)
Called when the task dies before completion, and we want to report back
diagnostic info
|
protected void |
Task.reportFatalError(TaskAttemptID id,
Throwable throwable,
String logMsg)
Report a fatal error to the parent (task) tracker.
|
void |
TaskUmbilicalProtocol.reportNextRecordRange(TaskAttemptID taskid,
org.apache.hadoop.mapred.SortedRanges.Range range,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report the record range which is going to process next by the Task.
|
void |
TaskTracker.reportNextRecordRange(TaskAttemptID taskid,
org.apache.hadoop.mapred.SortedRanges.Range range,
org.apache.hadoop.mapred.JvmContext jvmContext) |
void |
TaskReport.setSuccessfulAttempt(TaskAttemptID t)
set successful attempt ID of the task.
|
void |
TaskCompletionEvent.setTaskID(TaskAttemptID taskId)
Sets task id.
|
void |
TaskUmbilicalProtocol.shuffleError(TaskAttemptID taskId,
String message,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report that a reduce-task couldn't shuffle map-outputs.
|
void |
TaskTracker.shuffleError(TaskAttemptID taskId,
String message,
org.apache.hadoop.mapred.JvmContext jvmContext)
A reduce-task failed to shuffle the map-outputs.
|
boolean |
TaskUmbilicalProtocol.statusUpdate(TaskAttemptID taskId,
TaskStatus taskStatus,
org.apache.hadoop.mapred.JvmContext jvmContext)
Report child's progress to parent.
|
boolean |
TaskTracker.statusUpdate(TaskAttemptID taskid,
TaskStatus taskStatus,
org.apache.hadoop.mapred.JvmContext jvmContext)
Called periodically to report Task progress, from 0.0 to 1.0.
|
static void |
TaskLog.syncLogs(String logLocation,
TaskAttemptID taskid,
boolean isCleanup,
boolean segmented) |
void |
TaskTrackerMetricsSource.taskFailedPing(TaskAttemptID t) |
void |
TaskTrackerMetricsSource.timedoutTask(TaskAttemptID t) |
限定符和类型 | 方法和说明 |
---|---|
void |
TaskReport.setRunningTaskAttempts(Collection<TaskAttemptID> runningAttempts)
set running attempt(s) of the task.
|
构造器和说明 |
---|
Task(String jobFile,
TaskAttemptID taskId,
int partition,
int numSlotsRequired) |
TaskCompletionEvent(int eventId,
TaskAttemptID taskId,
int idWithinJob,
boolean isMap,
TaskCompletionEvent.Status status,
String taskTrackerHttp)
Constructor. eventId should be created externally and incremented
per event for each job.
|
TaskStatus(TaskAttemptID taskid,
float progress,
int numSlots,
TaskStatus.State runState,
String diagnosticInfo,
String stateString,
String taskTracker,
TaskStatus.Phase phase,
Counters counters) |
Copyright © 2009 The Apache Software Foundation