org.apache.hadoop.mapred.join

类 CompositeInputFormat<K extends WritableComparable>

java.lang.Object

org.apache.hadoop.mapred.join.CompositeInputFormat<K>

所有已实现的接口:

InputFormat<K,TupleWritable>, ComposableInputFormat<K,TupleWritable>

public class CompositeInputFormat<K extends WritableComparable>
extends Object
implements ComposableInputFormat<K,TupleWritable>

An InputFormat capable of performing joins over a set of data sources sorted and partitioned the same way.

另请参阅:

A user may define new join types by setting the property mapred.join.define.<ident> to a classname. In the expression mapred.join.expr, the identifier will be assumed to be a ComposableRecordReader. mapred.join.keycomparator can be a classname used to compare keys in the join., JoinRecordReader, MultiFilterRecordReader

构造器概要

构造器

构造器和说明
CompositeInputFormat()

方法概要

方法

限定符和类型	方法和说明
protected void	addDefaults() Adds the default set of identifiers to the parser.
static String	compose(Class<? extends InputFormat> inf, String path) Convenience method for constructing composite formats.
static String	compose(String op, Class<? extends InputFormat> inf, Path... path) Convenience method for constructing composite formats.
static String	compose(String op, Class<? extends InputFormat> inf, String... path) Convenience method for constructing composite formats.
ComposableRecordReader<K,TupleWritable>	getRecordReader(InputSplit split, JobConf job, Reporter reporter) Construct a CompositeRecordReader for the children of this InputFormat as defined in the init expression.
InputSplit[]	getSplits(JobConf job, int numSplits) Build a CompositeInputSplit from the child InputFormats by assigning the ith split from each child to the ith composite split.
void	setFormat(JobConf job) Interpret a given string as a composite expression.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

CompositeInputFormat

public CompositeInputFormat()

方法详细资料

setFormat

public void setFormat(JobConf job)
               throws IOException

Interpret a given string as a composite expression. func ::= <ident>([<func>,]*<func>) func ::= tbl(<class>,"<path>") class ::= @see java.lang.Class#forName(java.lang.String) path ::= @see org.apache.hadoop.fs.Path#Path(java.lang.String) Reads expression from the mapred.join.expr property and user-supplied join types from mapred.join.define.<ident> types. Paths supplied to tbl are given as input paths to the InputFormat class listed.

抛出:

IOException

另请参阅:

compose(java.lang.String, java.lang.Class, java.lang.String...)

addDefaults

protected void addDefaults()

Adds the default set of identifiers to the parser.

getSplits

public InputSplit[] getSplits(JobConf job,
                     int numSplits)
                       throws IOException

Build a CompositeInputSplit from the child InputFormats by assigning the ith split from each child to the ith composite split.

指定者:

getSplits 在接口中 InputFormat<K extends WritableComparable,TupleWritable>

参数:

job - job configuration.

numSplits - the desired number of splits, a hint.

返回:

an array of InputSplits for the job.

抛出:

IOException

getRecordReader

public ComposableRecordReader<K,TupleWritable> getRecordReader(InputSplit split,
                                                      JobConf job,
                                                      Reporter reporter)
                                                                                   throws IOException

Construct a CompositeRecordReader for the children of this InputFormat as defined in the init expression. The outermost join need only be composable, not necessarily a composite. Mandating TupleWritable isn't strictly correct.

指定者:

getRecordReader 在接口中 InputFormat<K extends WritableComparable,TupleWritable>

指定者:

getRecordReader 在接口中 ComposableInputFormat<K extends WritableComparable,TupleWritable>

参数:

split - the InputSplit

job - the job that this split belongs to

返回:

a RecordReader

抛出:

IOException

compose

public static String compose(Class<? extends InputFormat> inf,
             String path)

Convenience method for constructing composite formats. Given InputFormat class (inf), path (p) return: tbl(<inf>, <p>)

compose

public static String compose(String op,
             Class<? extends InputFormat> inf,
             String... path)

Convenience method for constructing composite formats. Given operation (op), Object class (inf), set of paths (p) return: <op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))

compose

public static String compose(String op,
             Class<? extends InputFormat> inf,
             Path... path)

Convenience method for constructing composite formats. Given operation (op), Object class (inf), set of paths (p) return: <op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))