| 程序包 | 说明 |
|---|---|
| org.apache.hadoop.contrib.index.mapred | |
| org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
| org.apache.hadoop.fs |
An abstract file system API.
|
| org.apache.hadoop.fs.ftp | |
| org.apache.hadoop.fs.kfs |
A client for the Kosmos filesystem (KFS)
Introduction
This pages describes how to use Kosmos Filesystem
( KFS ) as a backing
store with Hadoop.
|
| org.apache.hadoop.fs.s3 |
A distributed, block-based implementation of
FileSystem that uses Amazon S3
as a backing store. |
| org.apache.hadoop.fs.s3native |
A distributed implementation of
FileSystem for reading and writing files on
Amazon S3. |
| org.apache.hadoop.hdfs |
A distributed implementation of
FileSystem. |
| org.apache.hadoop.hdfs.web | |
| org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
| org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
| org.apache.hadoop.mapred.lib |
Library of generally useful mappers, reducers, and partitioners.
|
| org.apache.hadoop.mapred.lib.db |
org.apache.hadoop.mapred.lib.db Package
This package contains a library to read records from a database as an
input to a mapreduce job, and write the output records to the database.
|
| org.apache.hadoop.mapreduce | |
| org.apache.hadoop.util |
Common utilities.
|
| 限定符和类型 | 方法和说明 |
|---|---|
RecordWriter<Shard,Text> |
IndexUpdateOutputFormat.getRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
RecordWriter<Text,Text> |
TeraOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
RawLocalFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FilterFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
abstract FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
ChecksumFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
HarFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FilterFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
abstract FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
ChecksumFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
已过时。
API only for 0.20-append
|
FSDataOutputStream |
RawLocalFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
已过时。
API only for 0.20-append
|
FSDataOutputStream |
ChecksumFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
FTPFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
FTPFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
A stream obtained via this call must be closed before using other APIs of
this class or else the invocation will block.
|
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
KosmosFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
KosmosFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
S3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
S3FileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
NativeS3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
NativeS3FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
HftpFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
DistributedFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
DFSClient.append(String src,
int buffersize,
Progressable progress,
FileSystem.Statistics statistics)
Append to an existing HDFS file.
|
FSDataOutputStream |
HftpFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
DistributedFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
OutputStream |
DFSClient.create(String src,
boolean overwrite,
Progressable progress)
Create a new dfs file and return an output stream for writing into it
with write-progress reporting.
|
OutputStream |
DFSClient.create(String src,
boolean overwrite,
short replication,
long blockSize,
Progressable progress)
Create a new dfs file with the specified block replication
with write-progress reporting and return an output stream for writing
into the file.
|
OutputStream |
DFSClient.create(String src,
boolean overwrite,
short replication,
long blockSize,
Progressable progress,
int buffersize)
Call
DFSClient.create(String,FsPermission,boolean,short,long,Progressable,int)
with default permission. |
OutputStream |
DFSClient.create(String src,
FsPermission permission,
boolean overwrite,
boolean createParent,
short replication,
long blockSize,
Progressable progress,
int buffersize)
Create a new dfs file with the specified block replication
with write-progress reporting and return an output stream for writing
into the file.
|
OutputStream |
DFSClient.create(String src,
FsPermission permission,
boolean overwrite,
short replication,
long blockSize,
Progressable progress,
int buffersize)
Call
DFSClient.create(String,FsPermission,boolean,boolean,short,long,Progressable,int)
with createParent set to true. |
FSDataOutputStream |
DistributedFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Same as create(), except fails if parent directory doesn't already exist.
|
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
WebHdfsFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
FSDataOutputStream |
WebHdfsFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
| 限定符和类型 | 方法和说明 |
|---|---|
SequenceFile.Writer |
SequenceFile.Sorter.cloneFileAttributes(Path inputFile,
Path outputFile,
Progressable prog)
Clones the attributes (like compression of the input file and creates a
corresponding Writer
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
void |
SequenceFile.Sorter.setProgressable(Progressable progressable)
Set the progressable object in order to report progress.
|
| 限定符和类型 | 接口和说明 |
|---|---|
interface |
Reporter
A facility for Map-Reduce applications to report progress and update
counters, status information etc.
|
| 限定符和类型 | 类和说明 |
|---|---|
protected class |
Task.TaskReporter |
| 限定符和类型 | 方法和说明 |
|---|---|
Progressable |
TaskAttemptContext.getProgressible() |
Progressable |
JobContext.getProgressible()
Get the progress mechanism for reporting progress.
|
| 限定符和类型 | 方法和说明 |
|---|---|
RecordWriter<K,V> |
TextOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
SequenceFileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<BytesWritable,BytesWritable> |
SequenceFileAsBinaryOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
OutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress)
Get the
RecordWriter for the given job. |
RecordWriter<WritableComparable,Writable> |
MapFileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
abstract RecordWriter<K,V> |
FileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
| 构造器和说明 |
|---|
Task.CombineOutputCollector(Counters.Counter outCounter,
Progressable progressable,
Configuration conf) |
| 限定符和类型 | 方法和说明 |
|---|---|
protected RecordWriter<K,V> |
MultipleTextOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
protected RecordWriter<K,V> |
MultipleSequenceFileOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
protected abstract RecordWriter<K,V> |
MultipleOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
RecordWriter<K,V> |
NullOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
MultipleOutputFormat.getRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3)
Create a composite record writer that can write key/value data to different
output files
|
| 限定符和类型 | 方法和说明 |
|---|---|
RecordWriter<K,V> |
DBOutputFormat.getRecordWriter(FileSystem filesystem,
JobConf job,
String name,
Progressable progress)
Get the
RecordWriter for the given job. |
| 限定符和类型 | 类和说明 |
|---|---|
class |
MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper. |
class |
Mapper.Context |
class |
ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer. |
class |
Reducer.Context |
class |
TaskAttemptContext
The context for task attempts.
|
class |
TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
| 限定符和类型 | 字段和说明 |
|---|---|
protected Progressable |
ReduceContext.reporter |
| 限定符和类型 | 方法和说明 |
|---|---|
void |
QuickSort.sort(IndexedSortable s,
int p,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int), but indicate progress
periodically. |
void |
IndexedSorter.sort(IndexedSortable s,
int l,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int), but indicate progress
periodically. |
void |
HeapSort.sort(IndexedSortable s,
int p,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int), but indicate progress
periodically. |
Copyright © 2009 The Apache Software Foundation