org.apache.hadoop.tools.distcp2.mapred

类 CopyOutputFormat<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.TextOutputFormat<K,V>

org.apache.hadoop.tools.distcp2.mapred.CopyOutputFormat<K,V>

类型参数:

K -

V -

public class CopyOutputFormat<K,V>
extends TextOutputFormat<K,V>

The CopyOutputFormat is the Hadoop OutputFormat used in DistCp. It sets up the Job's Configuration (in the Job-Context) with the settings for the work-directory, final commit-directory, etc. It also sets the right output-committer.

嵌套类概要

从类继承的嵌套类/接口 org.apache.hadoop.mapreduce.lib.output.TextOutputFormat

TextOutputFormat.LineRecordWriter<K,V>

从类继承的嵌套类/接口 org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

FileOutputFormat.Counter

字段概要

从类继承的字段 org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, PART

构造器概要

构造器

构造器和说明
CopyOutputFormat()

方法概要

方法

限定符和类型	方法和说明
void	checkOutputSpecs(JobContext context) Check for validity of the output-specification for the job.
static Path	getCommitDirectory(Job job) Getter for the final commit-directory.
OutputCommitter	getOutputCommitter(TaskAttemptContext context) Get the output committer for this output format.
static Path	getWorkingDirectory(Job job) Getter for the working directory.
static void	setCommitDirectory(Job job, Path commitDirectory) Setter for the final directory for DistCp (where files copied will be moved, atomically.)
static void	setWorkingDirectory(Job job, Path workingDirectory) Setter for the working directory for DistCp (where files will be copied before they are moved to the final commit-directory.)

从类继承的方法 org.apache.hadoop.mapreduce.lib.output.TextOutputFormat

getRecordWriter

从类继承的方法 org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

CopyOutputFormat

public CopyOutputFormat()

方法详细资料

setWorkingDirectory

public static void setWorkingDirectory(Job job,
                       Path workingDirectory)

Setter for the working directory for DistCp (where files will be copied before they are moved to the final commit-directory.)

参数:

job - The Job on whose configuration the working-directory is to be set.

workingDirectory - The path to use as the working directory.

setCommitDirectory

public static void setCommitDirectory(Job job,
                      Path commitDirectory)

Setter for the final directory for DistCp (where files copied will be moved, atomically.)

参数:

job - The Job on whose configuration the working-directory is to be set.

commitDirectory - The path to use for final commit.

getWorkingDirectory

public static Path getWorkingDirectory(Job job)

Getter for the working directory.

参数:

job - The Job from whose configuration the working-directory is to be retrieved.

返回:

The working-directory Path.

getCommitDirectory

public static Path getCommitDirectory(Job job)

Getter for the final commit-directory.

参数:

job - The Job from whose configuration the commit-directory is to be retrieved.

返回:

The commit-directory Path.

getOutputCommitter

public OutputCommitter getOutputCommitter(TaskAttemptContext context)
                                   throws IOException

从类复制的说明: OutputFormat

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

覆盖:

getOutputCommitter 在类中 FileOutputFormat<K,V>

参数:

context - the task context

返回:

an output committer

抛出:

IOException

checkOutputSpecs

public void checkOutputSpecs(JobContext context)
                      throws IOException

从类复制的说明: OutputFormat

Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

覆盖:

checkOutputSpecs 在类中 FileOutputFormat<K,V>

参数:

context - information about the job

抛出:

IOException - when output should not be attempted