org.apache.hadoop.mapreduce.lib.input

类 CombineFileSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

所有已实现的接口:

Writable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class CombineFileSplit
extends InputSplit
implements Writable

A sub-collection of input files. Unlike FileSplit, CombineFileSplit class does not represent a split of a file, but a split of input files into smaller sets. A split may contain blocks from different file but all the blocks in the same split are probably local to some rack
CombineFileSplit can be used to implement RecordReader's, with reading one record per file.

另请参阅:

FileSplit, CombineFileInputFormat

构造器概要

构造器

构造器和说明
CombineFileSplit() default constructor
CombineFileSplit(CombineFileSplit old) Copy constructor
CombineFileSplit(Path[] files, long[] lengths)
CombineFileSplit(Path[] files, long[] start, long[] lengths, String[] locations)

方法概要

方法

限定符和类型	方法和说明
long	getLength() Get the size of the split, so that the input splits can be sorted by size.
long	getLength(int i) Returns the length of the ith Path
long[]	getLengths() Returns an array containing the lengths of the files in the split
String[]	getLocations() Returns all the Paths where this input-split resides
int	getNumPaths() Returns the number of Paths in the split
long	getOffset(int i) Returns the start offset of the ith Path
Path	getPath(int i) Returns the ith Path
Path[]	getPaths() Returns all the Paths in the split
long[]	getStartOffsets() Returns an array containing the start offsets of the files in the split
void	readFields(DataInput in) Deserialize the fields of this object from in.
String	toString()
void	write(DataOutput out) Serialize the fields of this object to out.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

构造器详细资料

CombineFileSplit

public CombineFileSplit()

default constructor

CombineFileSplit

public CombineFileSplit(Path[] files,
                long[] start,
                long[] lengths,
                String[] locations)

CombineFileSplit

public CombineFileSplit(Path[] files,
                long[] lengths)

CombineFileSplit

public CombineFileSplit(CombineFileSplit old)
                 throws IOException

Copy constructor

抛出:

IOException

方法详细资料

getLength

public long getLength()

从类复制的说明: InputSplit

Get the size of the split, so that the input splits can be sorted by size.

指定者:

getLength 在类中 InputSplit

返回:

the number of bytes in the split

getStartOffsets

public long[] getStartOffsets()

Returns an array containing the start offsets of the files in the split

getLengths

public long[] getLengths()

Returns an array containing the lengths of the files in the split

getOffset

public long getOffset(int i)

Returns the start offset of the i^th Path

getLength

public long getLength(int i)

Returns the length of the i^th Path

getNumPaths

public int getNumPaths()

Returns the number of Paths in the split

getPath

public Path getPath(int i)

Returns the i^th Path

getPaths

public Path[] getPaths()

Returns all the Paths in the split

getLocations

public String[] getLocations()
                      throws IOException

Returns all the Paths where this input-split resides

指定者:

getLocations 在类中 InputSplit

返回:

a new array of the node nodes.

抛出:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

从接口复制的说明: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

指定者:

readFields 在接口中 Writable

参数:

in - DataInput to deseriablize this object from.

抛出:

IOException

write

public void write(DataOutput out)
           throws IOException

从接口复制的说明: Writable

Serialize the fields of this object to out.

指定者:

write 在接口中 Writable

参数:

out - DataOuput to serialize this object into.

抛出:

IOException

toString

public String toString()

覆盖:

toString 在类中 Object