| 程序包 | 说明 |
|---|---|
| org.apache.hadoop.contrib.index.example | |
| org.apache.hadoop.contrib.index.mapred | |
| org.apache.hadoop.examples |
Hadoop example code.
|
| org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
| org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
| org.apache.hadoop.mapred.join |
Given a set of sorted datasets keyed with the same class and yielding equal
partitions, it is possible to effect a join of those datasets prior to the map.
|
| org.apache.hadoop.mapred.lib |
Library of generally useful mappers, reducers, and partitioners.
|
| org.apache.hadoop.mapred.lib.db |
org.apache.hadoop.mapred.lib.db Package
This package contains a library to read records from a database as an
input to a mapreduce job, and write the output records to the database.
|
| org.apache.hadoop.streaming |
Hadoop Streaming is a utility which allows users to create and run
Map-Reduce jobs with any executables (e.g.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
LineDocInputFormat
An InputFormat for LineDoc for plain text files where each line is a doc.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Class<? extends InputFormat> |
IndexUpdateConfiguration.getIndexInputFormatClass()
Get the index input format class.
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
IndexUpdateConfiguration.setIndexInputFormatClass(Class<? extends InputFormat> theClass)
Set the index input format class.
|
| 限定符和类型 | 类和说明 |
|---|---|
static class |
MultiFileWordCount.MyInputFormat
To use
MultiFileInputFormat, one should extend it, to return a
(custom) RecordReader. |
static class |
SleepJob.SleepInputFormat |
| 限定符和类型 | 类和说明 |
|---|---|
class |
TeraInputFormat
An input format that reads the first 10 characters of each line as the key
and the rest of the line as the value.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
FileInputFormat<K,V>
A base class for file-based
InputFormat. |
class |
KeyValueTextInputFormat
An
InputFormat for plain text files. |
class |
MultiFileInputFormat<K,V>
已过时。
Use
CombineFileInputFormat instead |
class |
SequenceFileAsBinaryInputFormat
InputFormat reading keys, values from SequenceFiles in binary (raw)
format.
|
class |
SequenceFileAsTextInputFormat
This class is similar to SequenceFileInputFormat, except it generates SequenceFileAsTextRecordReader
which converts the input keys and values to their String forms by calling toString() method.
|
class |
SequenceFileInputFilter<K,V>
A class that allows a map/red job to work on a sample of sequence files.
|
class |
SequenceFileInputFormat<K,V>
An
InputFormat for SequenceFiles. |
class |
TextInputFormat
An
InputFormat for plain text files. |
| 限定符和类型 | 方法和说明 |
|---|---|
InputFormat |
JobConf.getInputFormat()
Get the
InputFormat implementation for the map-reduce job,
defaults to TextInputFormat if not specified explicity. |
| 限定符和类型 | 方法和说明 |
|---|---|
void |
JobConf.setInputFormat(Class<? extends InputFormat> theClass)
Set the
InputFormat implementation for the map-reduce job. |
| 限定符和类型 | 接口和说明 |
|---|---|
interface |
ComposableInputFormat<K extends WritableComparable,V extends Writable>
Refinement of InputFormat requiring implementors to provide
ComposableRecordReader instead of RecordReader.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
CompositeInputFormat<K extends WritableComparable>
An InputFormat capable of performing joins over a set of data sources sorted
and partitioned the same way.
|
static class |
Parser.Node |
| 限定符和类型 | 方法和说明 |
|---|---|
static String |
CompositeInputFormat.compose(Class<? extends InputFormat> inf,
String path)
Convenience method for constructing composite formats.
|
static String |
CompositeInputFormat.compose(String op,
Class<? extends InputFormat> inf,
Path... path)
Convenience method for constructing composite formats.
|
static String |
CompositeInputFormat.compose(String op,
Class<? extends InputFormat> inf,
String... path)
Convenience method for constructing composite formats.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
CombineFileInputFormat<K,V>
|
class |
DelegatingInputFormat<K,V>
An
InputFormat that delegates behaviour of paths to multiple other
InputFormats. |
class |
NLineInputFormat
NLineInputFormat which splits N lines of input as one split.
|
| 限定符和类型 | 方法和说明 |
|---|---|
K[] |
InputSampler.Sampler.getSample(InputFormat<K,V> inf,
JobConf job)
For a given job, collect and return a subset of the keys from the
input data.
|
K[] |
InputSampler.SplitSampler.getSample(InputFormat<K,V> inf,
JobConf job)
From each split sampled, take the first numSamples / numSplits records.
|
K[] |
InputSampler.RandomSampler.getSample(InputFormat<K,V> inf,
JobConf job)
Randomize the split order, then take the specified number of keys from
each split sampled, where each key is selected with the specified
probability and possibly replaced by a subsequently selected key when
the quota of keys from that split is satisfied.
|
K[] |
InputSampler.IntervalSampler.getSample(InputFormat<K,V> inf,
JobConf job)
For each split sampled, emit when the ratio of the number of records
retained to the total record count is less than the specified
frequency.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
DBInputFormat<T extends DBWritable>
A InputFormat that reads input data from an SQL table.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
AutoInputFormat
An
InputFormat that tries to deduce the types of the input files
automatically. |
class |
StreamInputFormat
An input format that selects a RecordReader based on a JobConf property.
|
Copyright © 2009 The Apache Software Foundation