org.apache.hadoop.mapred

类 TaskLog

java.lang.Object

org.apache.hadoop.mapred.TaskLog

public class TaskLog
extends Object

A simple logger to handle the task-specific user logs. This class uses the system property hadoop.log.dir. This class is for Map/Reduce internal use only.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	TaskLog.LogName The filter for userlogs.

构造器概要

构造器

构造器和说明
TaskLog()

方法概要

方法

限定符和类型	方法和说明
static String	addCommand(List<String> cmd, boolean isExecutable) Add quotes to each of the command strings and return as a single string
static List<String>	captureDebugOut(List<String> cmd, File debugoutFilename) Wrap a command in a shell to capture debug script's stdout and stderr to debugout.
static List<String>	captureOutAndError(List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength) Wrap a command in a shell to capture stdout and stderr to files.
static List<String>	captureOutAndError(List<String> setup, List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength) Wrap a command in a shell to capture stdout and stderr to files.
static List<String>	captureOutAndError(List<String> setup, List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength, boolean useSetsid) Wrap a command in a shell to capture stdout and stderr to files.
static List<String>	captureOutAndError(List<String> setup, List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength, boolean useSetsid, String pidFileName) 已过时。 pidFiles are no more used. Instead pid is exported to env variable JVM_PID.
static List<String>	captureOutAndError(List<String> setup, List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength, String pidFileName) 已过时。 pidFiles are no more used. Instead pid is exported to env variable JVM_PID.
static void	createTaskAttemptLogDir(TaskAttemptID taskID, boolean isCleanup, String[] localDirs) Create log directory for the given attempt.
static String	getBaseLogDir()
static File	getJobDir(JobID jobid) Get the user log directory for the job jobid.
static File	getJobDir(String jobid) Get the user log directory for the job jobid.
static String	getTaskAttemptLogDir(TaskAttemptID taskID, String cleanupSuffix, String[] localDirs) Get attempt log directory path for the given attempt-id under randomly selected mapred local directory.
static File	getTaskLogFile(TaskAttemptID taskid, boolean isCleanup, TaskLog.LogName filter)
static long	getTaskLogLength(JobConf conf) Get the desired maximum length of task's logs.
static File	getUserLogDir()
static void	syncLogs(String logLocation, TaskAttemptID taskid, boolean isCleanup, boolean segmented)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

TaskLog

public TaskLog()

方法详细资料

createTaskAttemptLogDir

public static void createTaskAttemptLogDir(TaskAttemptID taskID,
                           boolean isCleanup,
                           String[] localDirs)
                                    throws IOException

Create log directory for the given attempt. This involves creating the following and setting proper permissions for the new directories
{hadoop.log.dir}/userlogs/
{hadoop.log.dir}/userlogs//
{one of the mapred-local-dirs}/userlogs/
{one of the mapred-local-dirs}/userlogs//

参数:

taskID - attempt-id for which log dir is to be created

isCleanup - Is this attempt a cleanup attempt ?

localDirs - mapred local directories

抛出:

IOException

getTaskAttemptLogDir

public static String getTaskAttemptLogDir(TaskAttemptID taskID,
                          String cleanupSuffix,
                          String[] localDirs)
                                   throws IOException

Get attempt log directory path for the given attempt-id under randomly selected mapred local directory.

参数:

taskID - attempt-id for which log dir path is needed

cleanupSuffix - ".cleanup" if this attempt is a cleanup attempt

localDirs - mapred local directories

返回:

target task attempt log directory

抛出:

IOException

getTaskLogFile

public static File getTaskLogFile(TaskAttemptID taskid,
                  boolean isCleanup,
                  TaskLog.LogName filter)

getBaseLogDir

public static String getBaseLogDir()

syncLogs

public static void syncLogs(String logLocation,
            TaskAttemptID taskid,
            boolean isCleanup,
            boolean segmented)
                     throws IOException

抛出:

IOException

getTaskLogLength

public static long getTaskLogLength(JobConf conf)

Get the desired maximum length of task's logs.

参数:

conf - the job to look in

返回:

the number of bytes to cap the log files at

captureOutAndError

public static List<String> captureOutAndError(List<String> cmd,
                              File stdoutFilename,
                              File stderrFilename,
                              long tailLength)
                                       throws IOException

Wrap a command in a shell to capture stdout and stderr to files. If the tailLength is 0, the entire output will be saved.

参数:

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

返回:

the modified command that should be run

抛出:

IOException

captureOutAndError

public static List<String> captureOutAndError(List<String> setup,
                              List<String> cmd,
                              File stdoutFilename,
                              File stderrFilename,
                              long tailLength)
                                       throws IOException

Wrap a command in a shell to capture stdout and stderr to files. Setup commands such as setting memory limit can be passed which will be executed before exec. If the tailLength is 0, the entire output will be saved.

参数:

setup - The setup commands for the execed process.

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

返回:

the modified command that should be run

抛出:

IOException

captureOutAndError

@Deprecated
public static List<String> captureOutAndError(List<String> setup,
                                         List<String> cmd,
                                         File stdoutFilename,
                                         File stderrFilename,
                                         long tailLength,
                                         String pidFileName)
                                       throws IOException

已过时。 pidFiles are no more used. Instead pid is exported to env variable JVM_PID.

Wrap a command in a shell to capture stdout and stderr to files. Setup commands such as setting memory limit can be passed which will be executed before exec. If the tailLength is 0, the entire output will be saved.

参数:

setup - The setup commands for the execed process.

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

返回:

the modified command that should be run

抛出:

IOException

captureOutAndError

@Deprecated
public static List<String> captureOutAndError(List<String> setup,
                                         List<String> cmd,
                                         File stdoutFilename,
                                         File stderrFilename,
                                         long tailLength,
                                         boolean useSetsid,
                                         String pidFileName)
                                       throws IOException

已过时。 pidFiles are no more used. Instead pid is exported to env variable JVM_PID.

Wrap a command in a shell to capture stdout and stderr to files. Setup commands such as setting memory limit can be passed which will be executed before exec. If the tailLength is 0, the entire output will be saved.

参数:

setup - The setup commands for the execed process.

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

useSetsid - Should setsid be used in the command or not.

返回:

the modified command that should be run

抛出:

IOException

captureOutAndError

public static List<String> captureOutAndError(List<String> setup,
                              List<String> cmd,
                              File stdoutFilename,
                              File stderrFilename,
                              long tailLength,
                              boolean useSetsid)
                                       throws IOException

Wrap a command in a shell to capture stdout and stderr to files. Setup commands such as setting memory limit can be passed which will be executed before exec. If the tailLength is 0, the entire output will be saved.

参数:

setup - The setup commands for the execed process.

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

useSetsid - Should setsid be used in the command or not.

返回:

the modified command that should be run

抛出:

IOException

addCommand

public static String addCommand(List<String> cmd,
                boolean isExecutable)
                         throws IOException

Add quotes to each of the command strings and return as a single string

参数:

cmd - The command to be quoted

isExecutable - makes shell path if the first argument is executable

返回:

returns The quoted string.

抛出:

IOException

captureDebugOut

public static List<String> captureDebugOut(List<String> cmd,
                           File debugoutFilename)
                                    throws IOException

Wrap a command in a shell to capture debug script's stdout and stderr to debugout.

参数:

cmd - The command and the arguments that should be run

debugoutFilename - The filename that stdout and stderr should be saved to.

返回:

the modified command that should be run

抛出:

IOException

getUserLogDir

public static File getUserLogDir()

getJobDir

public static File getJobDir(String jobid)

Get the user log directory for the job jobid.

参数:

jobid - string representation of the jobid

返回:

user log directory for the job

getJobDir

public static File getJobDir(JobID jobid)

Get the user log directory for the job jobid.

参数:

jobid - the jobid object

返回:

user log directory for the job