| 程序包 | 说明 |
|---|---|
| org.apache.hadoop.conf |
Configuration of system parameters.
|
| org.apache.hadoop.contrib.index.lucene | |
| org.apache.hadoop.contrib.index.mapred | |
| org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
| org.apache.hadoop.filecache | |
| org.apache.hadoop.fs |
An abstract file system API.
|
| org.apache.hadoop.fs.ftp | |
| org.apache.hadoop.fs.kfs |
A client for the Kosmos filesystem (KFS)
Introduction
This pages describes how to use Kosmos Filesystem
( KFS ) as a backing
store with Hadoop.
|
| org.apache.hadoop.fs.s3 |
A distributed, block-based implementation of
FileSystem that uses Amazon S3
as a backing store. |
| org.apache.hadoop.fs.s3native |
A distributed implementation of
FileSystem for reading and writing files on
Amazon S3. |
| org.apache.hadoop.fs.shell | |
| org.apache.hadoop.hdfs |
A distributed implementation of
FileSystem. |
| org.apache.hadoop.hdfs.protocol | |
| org.apache.hadoop.hdfs.web | |
| org.apache.hadoop.hdfs.web.resources | |
| org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
| org.apache.hadoop.io.compress | |
| org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
| org.apache.hadoop.mapred.join |
Given a set of sorted datasets keyed with the same class and yielding equal
partitions, it is possible to effect a join of those datasets prior to the map.
|
| org.apache.hadoop.mapred.lib |
Library of generally useful mappers, reducers, and partitioners.
|
| org.apache.hadoop.mapreduce | |
| org.apache.hadoop.mapreduce.lib.input | |
| org.apache.hadoop.mapreduce.lib.output | |
| org.apache.hadoop.mapreduce.lib.partition | |
| org.apache.hadoop.mapreduce.security | |
| org.apache.hadoop.mapreduce.split | |
| org.apache.hadoop.security | |
| org.apache.hadoop.tools | |
| org.apache.hadoop.tools.distcp2 | |
| org.apache.hadoop.tools.distcp2.mapred | |
| org.apache.hadoop.tools.distcp2.util | |
| org.apache.hadoop.tools.rumen |
Rumen is a data extraction and analysis tool built for
Apache Hadoop.
|
| org.apache.hadoop.util |
Common utilities.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
Configuration.getLocalPath(String dirsProp,
String path)
Get a local file under a directory named by dirsProp with
the given path.
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
Configuration.addResource(Path file)
Add a configuration resource.
|
| 构造器和说明 |
|---|
FileSystemDirectory(FileSystem fs,
Path directory,
boolean create,
Configuration conf)
Constructor
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
IndexUpdater.run(Configuration conf,
Path[] inputPaths,
Path outputPath,
int numMapTasks,
Shard[] shards) |
void |
IndexUpdater.run(Configuration conf,
Path[] inputPaths,
Path outputPath,
int numMapTasks,
Shard[] shards) |
void |
IIndexUpdater.run(Configuration conf,
Path[] inputPaths,
Path outputPath,
int numMapTasks,
Shard[] shards)
Create a Map/Reduce job configuration and run the Map/Reduce job to
analyze documents and update Lucene instances in parallel.
|
void |
IIndexUpdater.run(Configuration conf,
Path[] inputPaths,
Path outputPath,
int numMapTasks,
Shard[] shards)
Create a Map/Reduce job configuration and run the Map/Reduce job to
analyze documents and update Lucene instances in parallel.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
TeraInputFormat.writePartitionFile(JobConf conf,
Path partFile)
Use the input splits to take samples of the input and generate sample
keys.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path[] |
DistributedCache.getArchiveClassPaths(Configuration conf)
Get the archive entries in classpath as an array of Path.
|
static Path[] |
DistributedCache.getFileClassPaths(Configuration conf)
Get the file entries in classpath as an array of Path.
|
static Path[] |
DistributedCache.getLocalCacheArchives(Configuration conf)
Return the path array of the localized caches.
|
static Path[] |
DistributedCache.getLocalCacheFiles(Configuration conf)
Return the path array of the localized files.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
DistributedCache.addArchiveToClassPath(Path archive,
Configuration conf)
已过时。
Please use
DistributedCache.addArchiveToClassPath(Path, Configuration, FileSystem)
instead. The FileSystem should be obtained within an
appropriate doAs. |
static void |
DistributedCache.addArchiveToClassPath(Path archive,
Configuration conf,
FileSystem fs)
Add an archive path to the current set of classpath entries.
|
static void |
DistributedCache.addFileToClassPath(Path file,
Configuration conf)
已过时。
Please use
DistributedCache.addFileToClassPath(Path, Configuration, FileSystem)
instead. The FileSystem should be obtained within an
appropriate doAs. |
static void |
DistributedCache.addFileToClassPath(Path file,
Configuration conf,
FileSystem fs)
Add a file path to the current set of classpath entries.
|
static long |
TrackerDistributedCacheManager.downloadCacheObject(Configuration conf,
URI source,
Path destination,
long desiredTimestamp,
boolean isArchive,
FsPermission permission)
Download a given path to the local file system.
|
| 限定符和类型 | 字段和说明 |
|---|---|
protected Path |
FSInputChecker.file
The file name from which data is read from
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
ChecksumFileSystem.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
Path |
FsShell.getCurrentTrashDir()
Returns the Trash object associated with this shell.
|
Path[] |
InMemoryFileSystem.getFiles(PathFilter filter)
已过时。
|
Path |
RawLocalFileSystem.getHomeDirectory() |
Path |
HarFileSystem.getHomeDirectory()
return the top level archive path.
|
Path |
FilterFileSystem.getHomeDirectory() |
Path |
FileSystem.getHomeDirectory()
Return the current user's home directory in this filesystem.
|
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
Configuration conf)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
long size,
Configuration conf)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathForWrite(String pathStr,
long size,
Configuration conf,
boolean checkWrite)
Get a path from the local FS.
|
Path |
LocalDirAllocator.getLocalPathToRead(String pathStr,
Configuration conf)
Get a path from the local FS for reading.
|
Path |
Path.getParent()
Returns the parent of a path or null if at root.
|
Path |
FileStatus.getPath() |
Path |
RawLocalFileSystem.getWorkingDirectory() |
Path |
HarFileSystem.getWorkingDirectory()
return the top level archive.
|
Path |
FilterFileSystem.getWorkingDirectory()
Get the current working directory for the given file system
|
abstract Path |
FileSystem.getWorkingDirectory()
Get the current working directory for the given file system
|
Path |
Path.makeQualified(FileSystem fs)
Returns a qualified path object.
|
Path |
HarFileSystem.makeQualified(Path path) |
Path |
FilterFileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
FileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
Path.makeQualified(URI defaultUri,
Path workingDir)
Returns a qualified path object.
|
Path |
RawLocalFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
Path |
HarFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
Path |
FilterFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
FileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
ChecksumFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats)
convert an array of FileStatus to an array of Path
|
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats,
Path path)
convert an array of FileStatus to an array of Path.
|
Path |
Path.suffix(String suffix)
Adds a suffix to the final name in the path.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Iterable<Path> |
LocalDirAllocator.getAllLocalPathsToRead(String pathStr,
Configuration conf)
Get all of the paths that currently exist in the working directories.
|
| 限定符和类型 | 方法和说明 |
|---|---|
boolean |
PathFilter.accept(Path path)
Tests whether or not the specified abstract pathname should be
included in a pathname list.
|
FSDataOutputStream |
FileSystem.append(Path f)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize)
Append to an existing file (optional operation).
|
FSDataOutputStream |
RawLocalFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FilterFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
abstract FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
ChecksumFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
protected void |
FilterFileSystem.checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
protected void |
FileSystem.checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
RawLocalFileSystem.completeLocalOutput(Path fsWorkingFile,
Path tmpLocalFile) |
void |
HarFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
void |
FilterFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Called when we're all done writing to the target.
|
void |
FileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Called when we're all done writing to the target.
|
void |
ChecksumFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
void |
FilterFileSystem.concat(Path f,
Path[] psrcs) |
void |
FilterFileSystem.concat(Path f,
Path[] psrcs) |
void |
FileSystem.concat(Path trg,
Path[] srcs)
Concat existing files together.
|
void |
FileSystem.concat(Path trg,
Path[] srcs)
Concat existing files together.
|
static boolean |
FileUtil.copy(File src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
Configuration conf)
Copy local files to a FileSystem.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path[] srcs,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf) |
static boolean |
FileUtil.copy(FileSystem srcFS,
Path[] srcs,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf) |
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
File dst,
boolean deleteSource,
Configuration conf)
Copy FileSystem files to local files.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
boolean overwrite,
Configuration conf)
Copy files between FileSystems.
|
static boolean |
FileUtil.copy(FileSystem srcFS,
Path src,
FileSystem dstFS,
Path dst,
boolean deleteSource,
Configuration conf)
Copy files between FileSystems.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
The src file is on the local disk.
|
void |
LocalFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
HarFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
not implemented.
|
void |
FilterFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is on the local disk.
|
void |
FileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is on the local disk.
|
void |
ChecksumFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
FileSystem.copyFromLocalFile(Path src,
Path dst)
The src file is on the local disk.
|
static boolean |
FileUtil.copyMerge(FileSystem srcFS,
Path srcDir,
FileSystem dstFS,
Path dstFile,
boolean deleteSource,
Configuration conf,
String addString)
Copy all files in a directory to one output file (merge).
|
void |
LocalFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
HarFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
copies the file in the har filesystem to a local file.
|
void |
FilterFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
FileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
ChecksumFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
FileSystem.copyToLocalFile(Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
void |
ChecksumFileSystem.copyToLocalFile(Path src,
Path dst,
boolean copyCrc)
The src file is under FS, and the dst is on the local disk.
|
static FSDataOutputStream |
FileSystem.create(FileSystem fs,
Path file,
FsPermission permission)
create a file with the provided permission
The permission of the file is set to be the provided permission as in
setPermission, not permission&~umask
It is implemented using two RPCs.
|
FSDataOutputStream |
FileSystem.create(Path f)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
HarFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
FilterFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
abstract FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
ChecksumFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
HarFileSystem.create(Path f,
int bufferSize) |
FSDataOutputStream |
FileSystem.create(Path f,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication)
Opens an FSDataOutputStream at the indicated Path.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
boolean |
FileSystem.createNewFile(Path f)
Creates the given Path as a brand-new zero-length file.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
已过时。
API only for 0.20-append
|
FSDataOutputStream |
RawLocalFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
已过时。
API only for 0.20-append
|
FSDataOutputStream |
ChecksumFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
boolean |
RawLocalFileSystem.delete(Path p)
已过时。
|
boolean |
FilterFileSystem.delete(Path f)
已过时。
|
abstract boolean |
FileSystem.delete(Path f)
已过时。
Use delete(Path, boolean) instead
|
boolean |
RawLocalFileSystem.delete(Path p,
boolean recursive) |
boolean |
HarFileSystem.delete(Path f,
boolean recursive)
Not implemented.
|
boolean |
FilterFileSystem.delete(Path f,
boolean recursive)
Delete a file
|
abstract boolean |
FileSystem.delete(Path f,
boolean recursive)
Delete a file.
|
boolean |
ChecksumFileSystem.delete(Path f,
boolean recursive)
Implement the delete(Path, boolean) in checksum
file system.
|
boolean |
FileSystem.deleteOnExit(Path f)
Mark a path to be deleted when FileSystem is closed.
|
boolean |
LocalFileSystem.exists(Path f)
Get file status.
|
boolean |
FileSystem.exists(Path f)
Check if exists.
|
static void |
FileUtil.fullyDelete(FileSystem fs,
Path dir)
已过时。
|
long |
FileSystem.getBlockSize(Path f)
已过时。
Use getFileStatus() instead
|
Path |
ChecksumFileSystem.getChecksumFile(Path file)
Return the name of the checksum file associated with a file.
|
long |
ChecksumFileSystem.getChecksumFileLength(Path file,
long fileSize)
Return the length of the checksum file given the size of the
actual file.
|
ContentSummary |
FileSystem.getContentSummary(Path f)
Return the
ContentSummary of a given Path. |
long |
FileSystem.getDefaultBlockSize(Path f)
Return the number of bytes that large input files should be optimally
be split into to minimize i/o time.
|
short |
FileSystem.getDefaultReplication(Path path)
Get the default replication.
|
FileChecksum |
HarFileSystem.getFileChecksum(Path f) |
FileChecksum |
FilterFileSystem.getFileChecksum(Path f)
Get the checksum of a file.
|
FileChecksum |
FileSystem.getFileChecksum(Path f)
Get the checksum of a file.
|
FileStatus |
RawLocalFileSystem.getFileStatus(Path f) |
FileStatus |
HarFileSystem.getFileStatus(Path f)
return the filestatus of files in har archive.
|
FileStatus |
FilterFileSystem.getFileStatus(Path f)
Get file status.
|
abstract FileStatus |
FileSystem.getFileStatus(Path f)
Return a file status object that represents the path.
|
static int |
HarFileSystem.getHarHash(Path p)
the hash of the path p inside iniside
the filesystem
|
long |
FileSystem.getLength(Path f)
已过时。
Use getFileStatus() instead
|
short |
FileSystem.getReplication(Path src)
已过时。
Use getFileStatus() instead
|
FileStatus[] |
FileSystem.globStatus(Path pathPattern)
Return all the files that match filePattern and are not checksum
files.
|
FileStatus[] |
FileSystem.globStatus(Path pathPattern,
PathFilter filter)
Return an array of FileStatus objects whose path names match pathPattern
and is accepted by the user-supplied path filter.
|
static boolean |
ChecksumFileSystem.isChecksumFile(Path file)
Return true iff file is a checksum file name.
|
boolean |
FileSystem.isDirectory(Path f)
已过时。
Use getFileStatus() instead
|
boolean |
FileSystem.isFile(Path f)
True iff the named path is a regular file.
|
FileStatus[] |
RawLocalFileSystem.listStatus(Path f) |
FileStatus[] |
HarFileSystem.listStatus(Path f)
liststatus returns the children of a directory
after looking up the index files.
|
FileStatus[] |
FilterFileSystem.listStatus(Path f)
List files in a directory.
|
abstract FileStatus[] |
FileSystem.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
FileStatus[] |
ChecksumFileSystem.listStatus(Path f)
List the statuses of the files/directories in the given path if the path is
a directory.
|
FileStatus[] |
FileSystem.listStatus(Path[] files)
Filter files/directories in the given list of paths using default
path filter.
|
FileStatus[] |
FileSystem.listStatus(Path[] files,
PathFilter filter)
Filter files/directories in the given list of paths using user-supplied
path filter.
|
FileStatus[] |
FileSystem.listStatus(Path f,
PathFilter filter)
Filter files/directories in the given path using the user-supplied path
filter.
|
Path |
HarFileSystem.makeQualified(Path path) |
Path |
FilterFileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
FileSystem.makeQualified(Path path)
Make sure that a path specifies a FileSystem.
|
Path |
Path.makeQualified(URI defaultUri,
Path workingDir)
Returns a qualified path object.
|
static boolean |
FileSystem.mkdirs(FileSystem fs,
Path dir,
FsPermission permission)
create a directory with the provided permission
The permission of the directory is set to be the provided permission as in
setPermission, not permission&~umask
|
boolean |
RawLocalFileSystem.mkdirs(Path f)
Creates the specified directory hierarchy.
|
boolean |
FileSystem.mkdirs(Path f)
Call
FileSystem.mkdirs(Path, FsPermission) with default permission. |
boolean |
ChecksumFileSystem.mkdirs(Path f) |
boolean |
RawLocalFileSystem.mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
boolean |
HarFileSystem.mkdirs(Path f,
FsPermission permission)
not implemented.
|
boolean |
FilterFileSystem.mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
abstract boolean |
FileSystem.mkdirs(Path f,
FsPermission permission)
Make the given file and all non-existent parents into
directories.
|
void |
FileSystem.moveFromLocalFile(Path[] srcs,
Path dst)
The src files is on the local disk.
|
void |
FileSystem.moveFromLocalFile(Path[] srcs,
Path dst)
The src files is on the local disk.
|
void |
RawLocalFileSystem.moveFromLocalFile(Path src,
Path dst) |
void |
FileSystem.moveFromLocalFile(Path src,
Path dst)
The src file is on the local disk.
|
void |
FileSystem.moveToLocalFile(Path src,
Path dst)
The src file is under FS, and the dst is on the local disk.
|
boolean |
Trash.moveToTrash(Path path)
Move a file or directory to the current trash directory.
|
FSDataInputStream |
FileSystem.open(Path f)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
RawLocalFileSystem.open(Path f,
int bufferSize) |
FSDataInputStream |
HarFileSystem.open(Path f,
int bufferSize)
Returns a har input stream which fakes end of
file.
|
FSDataInputStream |
FilterFileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
abstract FSDataInputStream |
FileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
FSDataInputStream |
ChecksumFileSystem.open(Path f,
int bufferSize)
Opens an FSDataInputStream at the indicated Path.
|
File |
RawLocalFileSystem.pathToFile(Path path)
Convert a path to a File.
|
File |
LocalFileSystem.pathToFile(Path path)
Convert a path to a File.
|
boolean |
RawLocalFileSystem.rename(Path src,
Path dst) |
boolean |
FilterFileSystem.rename(Path src,
Path dst)
Renames Path src to Path dst.
|
abstract boolean |
FileSystem.rename(Path src,
Path dst)
Renames Path src to Path dst.
|
boolean |
ChecksumFileSystem.rename(Path src,
Path dst)
Rename files/dirs
|
boolean |
LocalFileSystem.reportChecksumFailure(Path p,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Moves files to a bad file directory on the same device, so that their
storage will not be reused.
|
boolean |
ChecksumFileSystem.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
Report a checksum error to the file system.
|
boolean |
InMemoryFileSystem.reserveSpaceWithCheckSum(Path f,
long size)
已过时。
Register a file with its size.
|
void |
RawLocalFileSystem.setOwner(Path p,
String username,
String groupname)
Use the command chown to set owner.
|
void |
HarFileSystem.setOwner(Path p,
String username,
String groupname)
not implemented.
|
void |
FilterFileSystem.setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e. a file or a directory).
|
void |
FileSystem.setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e. a file or a directory).
|
void |
RawLocalFileSystem.setPermission(Path p,
FsPermission permission)
Use the command chmod to set permission.
|
void |
HarFileSystem.setPermission(Path p,
FsPermission permisssion)
Not implemented.
|
void |
FilterFileSystem.setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
void |
FileSystem.setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
boolean |
HarFileSystem.setReplication(Path src,
short replication)
Not implemented.
|
boolean |
FilterFileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
boolean |
FileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
boolean |
ChecksumFileSystem.setReplication(Path src,
short replication)
Set replication for an existing file.
|
void |
FileSystem.setTimes(Path p,
long mtime,
long atime)
Set access time of a file
|
void |
RawLocalFileSystem.setWorkingDirectory(Path newDir)
Set the working directory to the given directory.
|
void |
HarFileSystem.setWorkingDirectory(Path newDir) |
void |
FilterFileSystem.setWorkingDirectory(Path newDir)
Set the current working directory for the given file system.
|
abstract void |
FileSystem.setWorkingDirectory(Path new_dir)
Set the current working directory for the given file system.
|
Path |
RawLocalFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
Path |
HarFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
Path |
FilterFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
FileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
Returns a local File that the user can write output to.
|
Path |
ChecksumFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
static Path[] |
FileUtil.stat2Paths(FileStatus[] stats,
Path path)
convert an array of FileStatus to an array of Path.
|
| 构造器和说明 |
|---|
FileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
long access_time,
FsPermission permission,
String owner,
String group,
Path path) |
FileStatus(long length,
boolean isdir,
int block_replication,
long blocksize,
long modification_time,
Path path) |
FSInputChecker(Path file,
int numOfRetries)
Constructor
|
FSInputChecker(Path file,
int numOfRetries,
boolean verifyChecksum,
Checksum sum,
int chunkSize,
int checksumSize)
Constructor
|
Path(Path parent,
Path child)
Resolve a child path against a parent path.
|
Path(Path parent,
String child)
Resolve a child path against a parent path.
|
Path(String parent,
Path child)
Resolve a child path against a parent path.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
FTPFileSystem.getHomeDirectory() |
Path |
FTPFileSystem.getWorkingDirectory() |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
FTPFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
FTPFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
A stream obtained via this call must be closed before using other APIs of
this class or else the invocation will block.
|
boolean |
FTPFileSystem.delete(Path file)
已过时。
Use delete(Path, boolean) instead
|
boolean |
FTPFileSystem.delete(Path file,
boolean recursive) |
FileStatus |
FTPFileSystem.getFileStatus(Path file) |
FileStatus[] |
FTPFileSystem.listStatus(Path file) |
boolean |
FTPFileSystem.mkdirs(Path file,
FsPermission permission) |
FSDataInputStream |
FTPFileSystem.open(Path file,
int bufferSize) |
boolean |
FTPFileSystem.rename(Path src,
Path dst) |
void |
FTPFileSystem.setWorkingDirectory(Path newDir) |
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
KosmosFileSystem.getWorkingDirectory() |
Path |
KosmosFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
KosmosFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
void |
KosmosFileSystem.completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
void |
KosmosFileSystem.copyFromLocalFile(boolean delSrc,
Path src,
Path dst) |
void |
KosmosFileSystem.copyToLocalFile(boolean delSrc,
Path src,
Path dst) |
FSDataOutputStream |
KosmosFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
KosmosFileSystem.delete(Path path)
已过时。
|
boolean |
KosmosFileSystem.delete(Path path,
boolean recursive) |
FileStatus |
KosmosFileSystem.getFileStatus(Path path) |
long |
KosmosFileSystem.getLength(Path path)
已过时。
|
short |
KosmosFileSystem.getReplication(Path path)
已过时。
|
boolean |
KosmosFileSystem.isDirectory(Path path)
已过时。
|
boolean |
KosmosFileSystem.isFile(Path path)
已过时。
|
FileStatus[] |
KosmosFileSystem.listStatus(Path path) |
void |
KosmosFileSystem.lock(Path path,
boolean shared)
已过时。
|
boolean |
KosmosFileSystem.mkdirs(Path path,
FsPermission permission) |
FSDataInputStream |
KosmosFileSystem.open(Path path,
int bufferSize) |
void |
KosmosFileSystem.release(Path path)
已过时。
|
boolean |
KosmosFileSystem.rename(Path src,
Path dst) |
boolean |
KosmosFileSystem.setReplication(Path path,
short replication) |
void |
KosmosFileSystem.setWorkingDirectory(Path dir) |
Path |
KosmosFileSystem.startLocalOutput(Path fsOutputFile,
Path tmpLocalFile) |
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
S3FileSystem.getWorkingDirectory() |
| 限定符和类型 | 方法和说明 |
|---|---|
Set<Path> |
FileSystemStore.listDeepSubPaths(Path path) |
Set<Path> |
FileSystemStore.listSubPaths(Path path) |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
S3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
S3FileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
S3FileSystem.delete(Path path)
已过时。
|
boolean |
S3FileSystem.delete(Path path,
boolean recursive) |
void |
FileSystemStore.deleteINode(Path path) |
FileStatus |
S3FileSystem.getFileStatus(Path f)
FileStatus for S3 file systems.
|
boolean |
FileSystemStore.inodeExists(Path path) |
boolean |
S3FileSystem.isFile(Path path) |
Set<Path> |
FileSystemStore.listDeepSubPaths(Path path) |
FileStatus[] |
S3FileSystem.listStatus(Path f) |
Set<Path> |
FileSystemStore.listSubPaths(Path path) |
boolean |
S3FileSystem.mkdirs(Path path,
FsPermission permission) |
FSDataInputStream |
S3FileSystem.open(Path path,
int bufferSize) |
boolean |
S3FileSystem.rename(Path src,
Path dst) |
INode |
FileSystemStore.retrieveINode(Path path) |
void |
S3FileSystem.setWorkingDirectory(Path dir) |
void |
FileSystemStore.storeINode(Path path,
INode inode) |
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
NativeS3FileSystem.getWorkingDirectory() |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
NativeS3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
NativeS3FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
NativeS3FileSystem.delete(Path path)
已过时。
|
boolean |
NativeS3FileSystem.delete(Path f,
boolean recurse) |
FileStatus |
NativeS3FileSystem.getFileStatus(Path f) |
FileStatus[] |
NativeS3FileSystem.listStatus(Path f)
If
f is a file, this method will make a single call to S3. |
boolean |
NativeS3FileSystem.mkdirs(Path f,
FsPermission permission) |
FSDataInputStream |
NativeS3FileSystem.open(Path f,
int bufferSize) |
boolean |
NativeS3FileSystem.rename(Path src,
Path dst) |
void |
NativeS3FileSystem.setWorkingDirectory(Path newDir)
Set the working directory to the given directory.
|
| 限定符和类型 | 方法和说明 |
|---|---|
protected void |
Count.run(Path path) |
protected abstract void |
Command.run(Path path)
Execute the command on the input path
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
DistributedFileSystem.getHomeDirectory()
Return the current user's home directory in this filesystem.
|
Path |
HftpFileSystem.getWorkingDirectory() |
Path |
DistributedFileSystem.getWorkingDirectory() |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
HftpFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
DistributedFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
void |
DistributedFileSystem.concat(Path trg,
Path[] psrcs)
Move blocks from srcs to trg and delete srcs afterwards.
|
void |
DistributedFileSystem.concat(Path trg,
Path[] psrcs)
Move blocks from srcs to trg and delete srcs afterwards.
|
FSDataOutputStream |
HftpFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
DistributedFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
FSDataOutputStream |
DistributedFileSystem.createNonRecursive(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Same as create(), except fails if parent directory doesn't already exist.
|
boolean |
HftpFileSystem.delete(Path f)
已过时。
|
boolean |
DistributedFileSystem.delete(Path f)
已过时。
|
boolean |
HftpFileSystem.delete(Path f,
boolean recursive) |
boolean |
DistributedFileSystem.delete(Path f,
boolean recursive)
requires a boolean check to delete a non
empty directory recursively.
|
ContentSummary |
HftpFileSystem.getContentSummary(Path f)
Return the
ContentSummary of a given Path. |
ContentSummary |
DistributedFileSystem.getContentSummary(Path f)
Return the
ContentSummary of a given Path. |
FileChecksum |
HftpFileSystem.getFileChecksum(Path f)
Get the checksum of a file.
|
MD5MD5CRC32FileChecksum |
DistributedFileSystem.getFileChecksum(Path f)
Get the checksum of a file.
|
FileStatus |
HftpFileSystem.getFileStatus(Path f) |
FileStatus |
DistributedFileSystem.getFileStatus(Path f)
Returns the stat information about the file.
|
FileStatus |
ChecksumDistributedFileSystem.getFileStatus(Path f)
Returns the stat information about the file.
|
boolean |
DistributedFileSystem.isFileClosed(Path src)
Get the close status of a file
|
FileStatus[] |
HftpFileSystem.listStatus(Path f) |
FileStatus[] |
DistributedFileSystem.listStatus(Path p)
List all the entries of a directory
Note that this operation is not atomic for a large directory.
|
boolean |
HftpFileSystem.mkdirs(Path f,
FsPermission permission) |
boolean |
DistributedFileSystem.mkdirs(Path f,
FsPermission permission) |
FSDataInputStream |
HftpFileSystem.open(Path f,
int buffersize) |
FSDataInputStream |
DistributedFileSystem.open(Path f,
int bufferSize) |
boolean |
DistributedFileSystem.recoverLease(Path f)
Start the lease recovery of a file
|
boolean |
HftpFileSystem.rename(Path src,
Path dst) |
boolean |
DistributedFileSystem.rename(Path src,
Path dst)
Rename files/dirs
|
boolean |
DistributedFileSystem.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
We need to find the blocks that didn't match.
|
boolean |
ChecksumDistributedFileSystem.reportChecksumFailure(Path f,
FSDataInputStream in,
long inPos,
FSDataInputStream sums,
long sumsPos)
We need to find the blocks that didn't match.
|
void |
DistributedFileSystem.setOwner(Path p,
String username,
String groupname)
Set owner of a path (i.e. a file or a directory).
|
void |
DistributedFileSystem.setPermission(Path p,
FsPermission permission)
Set permission of a path.
|
void |
DistributedFileSystem.setQuota(Path src,
long namespaceQuota,
long diskspaceQuota)
Set a directory's quotas
|
boolean |
DistributedFileSystem.setReplication(Path src,
short replication) |
void |
DistributedFileSystem.setTimes(Path p,
long mtime,
long atime)
Set access time of a file
|
void |
HftpFileSystem.setWorkingDirectory(Path f) |
void |
DistributedFileSystem.setWorkingDirectory(Path dir) |
| 构造器和说明 |
|---|
DFSClient.RemoteBlockReader(Path file,
int numRetries,
DataChecksum checksum,
boolean verifyChecksum) |
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
HdfsFileStatus.getFullPath(Path parent)
Get the full path
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
HdfsFileStatus.getFullPath(Path parent)
Get the full path
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
WebHdfsFileSystem.getHomeDirectory() |
Path |
WebHdfsFileSystem.getWorkingDirectory() |
| 限定符和类型 | 方法和说明 |
|---|---|
FSDataOutputStream |
WebHdfsFileSystem.append(Path f,
int bufferSize,
Progressable progress) |
void |
WebHdfsFileSystem.concat(Path trg,
Path[] srcs) |
void |
WebHdfsFileSystem.concat(Path trg,
Path[] srcs) |
FSDataOutputStream |
WebHdfsFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
boolean |
WebHdfsFileSystem.delete(Path f) |
boolean |
WebHdfsFileSystem.delete(Path f,
boolean recursive) |
ContentSummary |
WebHdfsFileSystem.getContentSummary(Path p) |
MD5MD5CRC32FileChecksum |
WebHdfsFileSystem.getFileChecksum(Path p) |
FileStatus |
WebHdfsFileSystem.getFileStatus(Path f) |
FileStatus[] |
WebHdfsFileSystem.listStatus(Path f) |
boolean |
WebHdfsFileSystem.mkdirs(Path f,
FsPermission permission) |
FSDataInputStream |
WebHdfsFileSystem.open(Path f,
int buffersize) |
boolean |
WebHdfsFileSystem.rename(Path src,
Path dst) |
void |
WebHdfsFileSystem.setOwner(Path p,
String owner,
String group) |
void |
WebHdfsFileSystem.setPermission(Path p,
FsPermission permission) |
boolean |
WebHdfsFileSystem.setReplication(Path p,
short replication) |
void |
WebHdfsFileSystem.setTimes(Path p,
long mtime,
long atime) |
void |
WebHdfsFileSystem.setWorkingDirectory(Path dir) |
| 构造器和说明 |
|---|
ConcatSourcesParam(Path[] paths) |
| 限定符和类型 | 方法和说明 |
|---|---|
SequenceFile.Writer |
SequenceFile.Sorter.cloneFileAttributes(Path inputFile,
Path outputFile,
Progressable prog)
Clones the attributes (like compression of the input file and creates a
corresponding Writer
|
protected SequenceFile.Reader |
MapFile.Reader.createDataFileReader(FileSystem fs,
Path dataFile,
Configuration conf)
Override this method to specialize the type of
SequenceFile.Reader returned. |
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
boolean createParent,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
static long |
MapFile.fix(FileSystem fs,
Path dir,
Class<? extends Writable> keyClass,
Class<? extends Writable> valueClass,
boolean dryrun,
Configuration conf)
This method attempts to fix a corrupt MapFile by re-creating its index.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(List<SequenceFile.Sorter.SegmentDescriptor> segments,
Path tmpDir)
Merges the list of segments of type
SegmentDescriptor |
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
int factor,
Path tmpDir)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
int factor,
Path tmpDir)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
Path tmpDir)
Merges the contents of files passed in Path[] using a max factor value
that is already set
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
boolean deleteInputs,
Path tmpDir)
Merges the contents of files passed in Path[] using a max factor value
that is already set
|
void |
SequenceFile.Sorter.merge(Path[] inFiles,
Path outFile)
Merge the provided files.
|
void |
SequenceFile.Sorter.merge(Path[] inFiles,
Path outFile)
Merge the provided files.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
Path tempDir,
boolean deleteInputs)
Merges the contents of files passed in Path[]
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.merge(Path[] inNames,
Path tempDir,
boolean deleteInputs)
Merges the contents of files passed in Path[]
|
protected FSDataInputStream |
SequenceFile.Reader.openFile(FileSystem fs,
Path file,
int bufferSize,
long length)
Override this method to specialize the type of
FSDataInputStream returned. |
void |
SequenceFile.Sorter.sort(Path[] inFiles,
Path outFile,
boolean deleteInput)
Perform a file sort from a set of input files into an output file.
|
void |
SequenceFile.Sorter.sort(Path[] inFiles,
Path outFile,
boolean deleteInput)
Perform a file sort from a set of input files into an output file.
|
void |
SequenceFile.Sorter.sort(Path inFile,
Path outFile)
The backwards compatible interface to sort.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.sortAndIterate(Path[] inFiles,
Path tempDir,
boolean deleteInput)
Perform a file sort from a set of input files and return an iterator.
|
SequenceFile.Sorter.RawKeyValueIterator |
SequenceFile.Sorter.sortAndIterate(Path[] inFiles,
Path tempDir,
boolean deleteInput)
Perform a file sort from a set of input files and return an iterator.
|
| 构造器和说明 |
|---|
SequenceFile.Reader(FileSystem fs,
Path file,
Configuration conf)
Open the named file.
|
SequenceFile.Sorter.SegmentDescriptor(long segmentOffset,
long segmentLength,
Path segmentPathName)
Constructs a segment
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass)
Create the named file.
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
SequenceFile.Metadata metadata)
Create the named file with write-progress reporter.
|
SequenceFile.Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
Progressable progress,
SequenceFile.Metadata metadata)
Create the named file with write-progress reporter.
|
| 限定符和类型 | 方法和说明 |
|---|---|
CompressionCodec |
CompressionCodecFactory.getCodec(Path file)
Find the relevant compression codec for the given file based on its
filename suffix.
|
| 限定符和类型 | 字段和说明 |
|---|---|
protected static Path |
JobHistory.DONE |
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
JobLocalizer.findCredentials() |
static Path[] |
FileInputFormat.getInputPaths(JobConf conf)
Get the list of input
Paths for the map-reduce job. |
static Path |
JobHistory.JobInfo.getJobHistoryLogLocation(String logFileName)
Get the job history file path given the history filename
|
static Path |
JobHistory.JobInfo.getJobHistoryLogLocationForUser(String logFileName,
JobConf jobConf)
Get the user job history file path
|
Path |
JobConf.getLocalPath(String pathString)
Constructs a local file name.
|
static Path |
FileOutputFormat.getOutputPath(JobConf conf)
Get the
Path to the output directory for the map-reduce job. |
Path |
FileSplit.getPath()
The file containing this split's data.
|
static Path |
FileOutputFormat.getPathForCustomFile(JobConf conf,
String name)
Helper function to generate a
Path for a file that is unique for
the task within the job output directory. |
static Path |
SkipBadRecords.getSkipOutputPath(Configuration conf)
Get the directory to which skipped records are written.
|
Path |
JobClient.getStagingAreaDir()
Grab the jobtracker's view of the staging directory path where
job-specific files will be placed.
|
Path |
JobClient.getSystemDir()
Grab the jobtracker system directory path where job-specific files are to be placed.
|
static Path |
FileOutputFormat.getTaskOutputPath(JobConf conf,
String name)
Helper function to create the task's temporary output directory and
return the path to the task's output file.
|
Path |
JobConf.getWorkingDirectory()
Get the current working directory for the default file system.
|
static Path |
FileOutputFormat.getWorkOutputPath(JobConf conf)
Get the
Path to the task's temporary output directory
for the map-reduce job
Tasks' Side-Effect Files
Note: The following is valid only if the |
static Path |
JobHistory.JobInfo.recoverJobHistoryFile(JobConf conf,
Path logFilePath)
Selects one of the two files generated as a part of recovery.
|
| 限定符和类型 | 方法和说明 |
|---|---|
boolean |
Utils.OutputFileUtils.OutputFilesFilter.accept(Path path) |
boolean |
Utils.OutputFileUtils.OutputLogFilter.accept(Path path) |
boolean |
OutputLogFilter.accept(Path path)
已过时。
|
static void |
FileInputFormat.addInputPath(JobConf conf,
Path path)
Add a
Path to the list of inputs for the map-reduce job. |
protected static FileSystem.Statistics |
Task.getFsStatistics(Path path,
Configuration conf)
Gets a handle to the Statistics instance based on the scheme associated
with path.
|
static SequenceFile.Reader[] |
SequenceFileOutputFormat.getReaders(Configuration conf,
Path dir)
Open the output generated by this format.
|
static MapFile.Reader[] |
MapFileOutputFormat.getReaders(FileSystem ignored,
Path dir,
Configuration conf)
Open the output generated by this format.
|
abstract void |
TaskController.initializeJob(String user,
String jobid,
Path credentials,
Path jobConf,
TaskUmbilicalProtocol taskTracker,
InetSocketAddress ttAddr)
Create all of the directories necessary for the job to start and download
all of the job and private distributed cache files.
|
void |
DefaultTaskController.initializeJob(String user,
String jobid,
Path credentials,
Path jobConf,
TaskUmbilicalProtocol taskTracker,
InetSocketAddress ttAddr)
This routine initializes the local file system for running a job.
|
static boolean |
JobClient.isJobDirValid(Path jobDirPath,
FileSystem fs)
Checks if the job directory is clean and has all the required components
for (re) starting the job
|
protected boolean |
TextInputFormat.isSplitable(FileSystem fs,
Path file) |
protected boolean |
KeyValueTextInputFormat.isSplitable(FileSystem fs,
Path file) |
protected boolean |
FileInputFormat.isSplitable(FileSystem fs,
Path filename)
Is the given filename splitable?
|
void |
JobLocalizer.localizeJobFiles(JobID jobid,
JobConf jConf,
Path localJobFile,
Path localJobTokenFile,
TaskUmbilicalProtocol taskTracker) |
void |
JobLocalizer.localizeJobFiles(JobID jobid,
JobConf jConf,
Path localJobTokenFile,
TaskUmbilicalProtocol taskTracker) |
static Path |
JobHistory.JobInfo.recoverJobHistoryFile(JobConf conf,
Path logFilePath)
Selects one of the two files generated as a part of recovery.
|
int |
JobLocalizer.runSetup(String user,
String jobid,
Path localJobTokenFile,
TaskUmbilicalProtocol taskTracker) |
static void |
FileInputFormat.setInputPaths(JobConf conf,
Path... inputPaths)
Set the array of
Paths as the list of inputs
for the map-reduce job. |
static void |
FileOutputFormat.setOutputPath(JobConf conf,
Path outputDir)
Set the
Path of the output directory for the map-reduce job. |
static void |
SkipBadRecords.setSkipOutputPath(JobConf conf,
Path path)
Set the directory to which skipped records are written.
|
void |
JobConf.setWorkingDirectory(Path dir)
Set the current working directory for the default file system.
|
protected static String |
TaskController.writeCommand(String cmdLine,
FileSystem fs,
Path commandFile) |
static void |
JobLocalizer.writeLocalJobFile(Path jobFile,
JobConf conf)
Write the task specific job-configuration file.
|
| 构造器和说明 |
|---|
FileSplit(Path file,
long start,
long length,
JobConf conf)
已过时。
|
FileSplit(Path file,
long start,
long length,
String[] hosts)
Constructs a split with host information
|
JobConf(Path config)
Construct a map/reduce configuration.
|
MultiFileSplit(JobConf job,
Path[] files,
long[] lengths)
已过时。
|
| 限定符和类型 | 方法和说明 |
|---|---|
static String |
CompositeInputFormat.compose(String op,
Class<? extends InputFormat> inf,
Path... path)
Convenience method for constructing composite formats.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
CombineFileSplit.getPath(int i)
Returns the ith Path
|
Path[] |
CombineFileSplit.getPaths()
Returns all the Paths in the split
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(JobConf conf,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
protected static FileSplit |
NLineInputFormat.createFileSplit(Path fileName,
long begin,
long length)
NLineInputFormat uses LineRecordReader, which always reads
(and consumes) at least one character out of its upper split
boundary.
|
static void |
TotalOrderPartitioner.setPartitionFile(JobConf job,
Path p)
Set the path to the SequenceFile storing the sorted partition keyset.
|
| 构造器和说明 |
|---|
CombineFileSplit(JobConf job,
Path[] files,
long[] lengths) |
CombineFileSplit(JobConf job,
Path[] files,
long[] start,
long[] lengths,
String[] locations) |
| 限定符和类型 | 方法和说明 |
|---|---|
static Path |
JobSubmissionFiles.getJobConfPath(Path jobSubmitDir)
Get the job conf path.
|
static Path |
JobSubmissionFiles.getJobDistCacheArchives(Path jobSubmitDir)
Get the job distributed cache archives path.
|
static Path |
JobSubmissionFiles.getJobDistCacheFiles(Path jobSubmitDir)
Get the job distributed cache files path.
|
static Path |
JobSubmissionFiles.getJobDistCacheLibjars(Path jobSubmitDir)
Get the job distributed cache libjars path.
|
static Path |
JobSubmissionFiles.getJobJar(Path jobSubmitDir)
Get the job jar path.
|
static Path |
JobSubmissionFiles.getJobSplitFile(Path jobSubmissionDir) |
static Path |
JobSubmissionFiles.getJobSplitMetaFile(Path jobSubmissionDir) |
static Path |
JobSubmissionFiles.getStagingDir(JobClient client,
Configuration conf)
Initializes the staging directory and returns the path.
|
Path |
JobContext.getWorkingDirectory()
Get the current working directory for the default file system.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path |
JobSubmissionFiles.getJobConfPath(Path jobSubmitDir)
Get the job conf path.
|
static Path |
JobSubmissionFiles.getJobDistCacheArchives(Path jobSubmitDir)
Get the job distributed cache archives path.
|
static Path |
JobSubmissionFiles.getJobDistCacheFiles(Path jobSubmitDir)
Get the job distributed cache files path.
|
static Path |
JobSubmissionFiles.getJobDistCacheLibjars(Path jobSubmitDir)
Get the job distributed cache libjars path.
|
static Path |
JobSubmissionFiles.getJobJar(Path jobSubmitDir)
Get the job jar path.
|
static Path |
JobSubmissionFiles.getJobSplitFile(Path jobSubmissionDir) |
static Path |
JobSubmissionFiles.getJobSplitMetaFile(Path jobSubmissionDir) |
void |
Job.setWorkingDirectory(Path dir)
Set the current working directory for the default file system.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path[] |
FileInputFormat.getInputPaths(JobContext context)
Get the list of input
Paths for the map-reduce job. |
Path |
FileSplit.getPath()
The file containing this split's data.
|
Path |
CombineFileSplit.getPath(int i)
Returns the ith Path
|
Path[] |
CombineFileSplit.getPaths()
Returns all the Paths in the split
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
FileInputFormat.addInputPath(Job job,
Path path)
Add a
Path to the list of inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass)
Add a
Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
static void |
MultipleInputs.addInputPath(Job job,
Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
|
protected static FileSplit |
NLineInputFormat.createFileSplit(Path fileName,
long begin,
long length)
NLineInputFormat uses LineRecordReader, which always reads
(and consumes) at least one character out of its upper split
boundary.
|
protected boolean |
TextInputFormat.isSplitable(JobContext context,
Path file) |
protected boolean |
KeyValueTextInputFormat.isSplitable(JobContext context,
Path file) |
protected boolean |
FileInputFormat.isSplitable(JobContext context,
Path filename)
Is the given filename splitable?
|
protected boolean |
CombineFileInputFormat.isSplitable(JobContext context,
Path file) |
static void |
FileInputFormat.setInputPaths(Job job,
Path... inputPaths)
Set the array of
Paths as the list of inputs
for the map-reduce job. |
| 构造器和说明 |
|---|
CombineFileSplit(Path[] files,
long[] lengths) |
CombineFileSplit(Path[] files,
long[] start,
long[] lengths,
String[] locations) |
FileSplit(Path file,
long start,
long length,
String[] hosts)
Constructs a split with host information
|
| 限定符和类型 | 方法和说明 |
|---|---|
Path |
FileOutputFormat.getDefaultWorkFile(TaskAttemptContext context,
String extension)
Get the default path and filename for the output format.
|
static Path |
FileOutputFormat.getOutputPath(JobContext job)
Get the
Path to the output directory for the map-reduce job. |
static Path |
FileOutputFormat.getPathForWorkFile(TaskInputOutputContext<?,?,?,?> context,
String name,
String extension)
Helper function to generate a
Path for a file that is unique for
the task within the job output directory. |
static Path |
FileOutputFormat.getWorkOutputPath(TaskInputOutputContext<?,?,?,?> context)
Get the
Path to the task's temporary output directory
for the map-reduce job
Tasks' Side-Effect Files Some applications need to create/write-to side-files, which differ from the actual job-outputs. |
Path |
FileOutputCommitter.getWorkPath()
Get the directory that the task should write results into
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
FileOutputFormat.setOutputPath(Job job,
Path outputDir)
Set the
Path of the output directory for the map-reduce job. |
| 构造器和说明 |
|---|
FileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
TotalOrderPartitioner.setPartitionFile(Configuration conf,
Path p)
Set the path to the SequenceFile storing the sorted partition keyset.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
TokenCache.obtainTokensForNamenodes(Credentials credentials,
Path[] ps,
Configuration conf)
Convenience method to obtain delegation tokens from namenodes
corresponding to the paths passed.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
InputSplit[] splits) |
static <T extends InputSplit> |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
List<InputSplit> splits) |
static <T extends InputSplit> |
JobSplitWriter.createSplitFiles(Path jobSubmitDir,
Configuration conf,
FileSystem fs,
T[] splits) |
static JobSplit.TaskSplitMetaInfo[] |
SplitMetaInfoReader.readSplitMetaInfo(JobID jobId,
FileSystem fs,
Configuration conf,
Path jobSubmitDir) |
| 限定符和类型 | 方法和说明 |
|---|---|
static Credentials |
Credentials.readTokenStorageFile(Path filename,
Configuration conf)
Convenience method for reading a token storage file, and loading the Tokens
therein in the passed UGI
|
void |
Credentials.writeTokenStorageFile(Path filename,
Configuration conf) |
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
DistCp.copy(Configuration conf,
String srcPath,
String destPath,
Path logPath,
boolean srcAsList,
boolean ignoreReadFailures)
已过时。
|
| 限定符和类型 | 方法和说明 |
|---|---|
protected Path |
DistCp.createInputFileListing(Job job)
Create input listing by invoking an appropriate copy listing
implementation.
|
Path |
DistCpOptions.getAtomicWorkPath()
Get work path for atomic commit.
|
protected Path |
DistCp.getFileListingPath()
Get default name of the copy listing file.
|
Path |
DistCpOptions.getLogPath()
Get output directory for writing distcp logs.
|
Path |
DistCpOptions.getSourceFileListing()
File path (hdfs:// or file://) that contains the list of actual
files to copy
|
Path |
DistCpOptions.getTargetPath()
Getter for the targetPath.
|
| 限定符和类型 | 方法和说明 |
|---|---|
List<Path> |
DistCpOptions.getSourcePaths()
Getter for sourcePaths.
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
CopyListing.buildListing(Path pathToListFile,
DistCpOptions options)
Build listing function creates the input listing that distcp uses to
perform the copy.
|
void |
SimpleCopyListing.doBuildListing(Path pathToListingFile,
DistCpOptions options)
The interface to be implemented by sub-classes, to create the source/target file listing.
|
void |
GlobbedCopyListing.doBuildListing(Path pathToListingFile,
DistCpOptions options)
Implementation of CopyListing::buildListing().
|
void |
FileBasedCopyListing.doBuildListing(Path pathToListFile,
DistCpOptions options)
Implementation of CopyListing::buildListing().
|
protected abstract void |
CopyListing.doBuildListing(Path pathToListFile,
DistCpOptions options)
The interface to be implemented by sub-classes, to create the source/target file listing.
|
void |
DistCpOptions.setAtomicWorkPath(Path atomicWorkPath)
Set the work path for atomic commit
|
void |
DistCpOptions.setLogPath(Path logPath)
Set the log path where distcp output logs are stored
Uses JobStagingDir/_logs by default
|
protected boolean |
SimpleCopyListing.shouldCopy(Path path,
DistCpOptions options)
Provide an option to skip copy of a path, Allows for exclusion
of files such as
FileOutputCommitter.SUCCEEDED_FILE_NAME |
| 限定符和类型 | 方法和说明 |
|---|---|
void |
DistCpOptions.setSourcePaths(List<Path> sourcePaths)
Setter for sourcePaths.
|
| 构造器和说明 |
|---|
DistCpOptions(List<Path> sourcePaths,
Path targetPath)
Constructor, to initialize source/target paths.
|
DistCpOptions(Path sourceFileListing,
Path targetPath)
Constructor, to initialize source/target paths.
|
| 构造器和说明 |
|---|
DistCpOptions(List<Path> sourcePaths,
Path targetPath)
Constructor, to initialize source/target paths.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path |
CopyOutputFormat.getCommitDirectory(Job job)
Getter for the final commit-directory.
|
static Path |
CopyOutputFormat.getWorkingDirectory(Job job)
Getter for the working directory.
|
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
CopyOutputFormat.setCommitDirectory(Job job,
Path commitDirectory)
Setter for the final directory for DistCp (where files copied will be
moved, atomically.)
|
static void |
CopyOutputFormat.setWorkingDirectory(Job job,
Path workingDirectory)
Setter for the working directory for DistCp (where files will be copied
before they are moved to the final commit-directory.)
|
| 构造器和说明 |
|---|
CopyCommitter(Path outputPath,
TaskAttemptContext context)
Create a output committer
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path |
DistCpUtils.sortListing(FileSystem fs,
Configuration conf,
Path sourceListing)
Sort sequence file containing FileStatus and Text as key and value respecitvely
|
| 限定符和类型 | 方法和说明 |
|---|---|
static boolean |
DistCpUtils.checksumsAreEqual(FileSystem sourceFS,
Path source,
FileSystem targetFS,
Path target)
Utility to compare checksums for the paths specified.
|
static long |
DistCpUtils.getFileSize(Path path,
Configuration configuration)
Retrieves size of the file at the specified path.
|
static String |
DistCpUtils.getRelativePath(Path sourceRootPath,
Path childPath)
Gets relative path of child path with respect to a root path
For ex.
|
static void |
DistCpUtils.preserve(FileSystem targetFS,
Path path,
FileStatus srcFileStatus,
EnumSet<DistCpOptions.FileAttribute> attributes)
Preserve attribute on file matching that of the file status being sent
as argument.
|
static Path |
DistCpUtils.sortListing(FileSystem fs,
Configuration conf,
Path sourceListing)
Sort sequence file containing FileStatus and Text as key and value respecitvely
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
InputDemuxer.bindTo(Path path,
Configuration conf)
Bind the
InputDemuxer to a particular file. |
void |
DefaultInputDemuxer.bindTo(Path path,
Configuration conf) |
void |
Outputter.init(Path path,
Configuration conf)
Initialize the
Outputter to a specific path. |
void |
DefaultOutputter.init(Path path,
Configuration conf) |
| 构造器和说明 |
|---|
ClusterTopologyReader(Path path,
Configuration conf)
Constructor.
|
JobTraceReader(Path path,
Configuration conf)
Constructor.
|
ZombieCluster(Path path,
MachineNode defaultNode,
Configuration conf)
Construct a homogeneous cluster.
|
ZombieJobProducer(Path path,
ZombieCluster cluster,
Configuration conf)
Constructor
|
ZombieJobProducer(Path path,
ZombieCluster cluster,
Configuration conf,
long randomSeed)
Constructor
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Path[] |
StringUtils.stringToPath(String[] str) |
| 限定符和类型 | 方法和说明 |
|---|---|
static void |
DiskChecker.checkDir(LocalFileSystem localFS,
Path dir,
FsPermission expected)
Create the local directory if necessary, check permissions and also ensure
it can be read from and written into.
|
static boolean |
DiskChecker.mkdirsWithExistsAndPermissionCheck(LocalFileSystem localFS,
Path dir,
FsPermission expected)
Create the directory or check permissions if it already exists.
|
Copyright © 2009 The Apache Software Foundation