public class DistCpOptions extends Object
限定符和类型 | 类和说明 |
---|---|
static class |
DistCpOptions.FileAttribute |
构造器和说明 |
---|
DistCpOptions(DistCpOptions that)
Copy constructor.
|
DistCpOptions(List<Path> sourcePaths,
Path targetPath)
Constructor, to initialize source/target paths.
|
DistCpOptions(Path sourceFileListing,
Path targetPath)
Constructor, to initialize source/target paths.
|
限定符和类型 | 方法和说明 |
---|---|
void |
appendToConf(Configuration conf)
Add options to configuration.
|
protected DistCpOptions |
clone() |
Path |
getAtomicWorkPath()
Get work path for atomic commit.
|
String |
getCopyStrategy()
Get the copy strategy to use.
|
Path |
getLogPath()
Get output directory for writing distcp logs.
|
int |
getMapBandwidth()
Get the map bandwidth in MB
|
int |
getMaxMaps()
Get the max number of maps to use for this copy
|
Path |
getSourceFileListing()
File path (hdfs:// or file://) that contains the list of actual
files to copy
|
List<Path> |
getSourcePaths()
Getter for sourcePaths.
|
String |
getSslConfigurationFile()
Get path where the ssl configuration file is present to use for hftps://
|
Path |
getTargetPath()
Getter for the targetPath.
|
void |
preserve(DistCpOptions.FileAttribute fileAttribute)
Add file attributes that need to be preserved.
|
Iterator<DistCpOptions.FileAttribute> |
preserveAttributes()
Returns an iterator with the list of file attributes to preserve
|
void |
setAtomicCommit(boolean atomicCommit)
Set if data need to be committed automatically
|
void |
setAtomicWorkPath(Path atomicWorkPath)
Set the work path for atomic commit
|
void |
setBlocking(boolean blocking)
Set if Disctp should run blocking or non-blocking
|
void |
setCopyStrategy(String copyStrategy)
Set the copy strategy to use.
|
void |
setDeleteMissing(boolean deleteMissing)
Set if files only present in target should be deleted
|
void |
setIgnoreFailures(boolean ignoreFailures)
Set if failures during copy be ignored
|
void |
setLogPath(Path logPath)
Set the log path where distcp output logs are stored
Uses JobStagingDir/_logs by default
|
void |
setMapBandwidth(int mapBandwidth)
Set per map bandwidth
|
void |
setMaxMaps(int maxMaps)
Set the max number of maps to use for copy
|
void |
setOverwrite(boolean overwrite)
Set if files should always be overwritten on target
|
void |
setSkipCRC(boolean skipCRC)
Set if checksum comparison should be skipped while determining if
source and destination files are identical
|
void |
setSourcePaths(List<Path> sourcePaths)
Setter for sourcePaths.
|
void |
setSslConfigurationFile(String sslConfigurationFile)
Set the SSL configuration file path to use with hftps:// (local path)
|
void |
setSyncFolder(boolean syncFolder)
Set if source and target folder contents be sync'ed up
|
boolean |
shouldAtomicCommit()
Should the data be committed atomically?
|
boolean |
shouldBlock()
Should DistCp be running in blocking mode
|
boolean |
shouldDeleteMissing()
Should target files missing in source should be deleted?
|
boolean |
shouldIgnoreFailures()
Should failures be logged and ignored during copy?
|
boolean |
shouldOverwrite()
Should files be overwritten always?
|
boolean |
shouldPreserve(DistCpOptions.FileAttribute attribute)
Checks if the input attibute should be preserved or not
|
boolean |
shouldSkipCRC()
Should CRC/checksum check be skipped while checking files are identical
|
boolean |
shouldSyncFolder()
Should the data be sync'ed between source and target paths?
|
String |
toString()
Utility to easily string-ify Options, for logging.
|
void |
validate(DistCpOptionSwitch option,
boolean value) |
public DistCpOptions(List<Path> sourcePaths, Path targetPath)
sourcePaths
- List of source-paths (including wildcards)
to be copied to target.targetPath
- Destination path for the dist-copy.public DistCpOptions(Path sourceFileListing, Path targetPath)
sourceFileListing
- File containing list of source pathstargetPath
- Destination path for the dist-copy.public DistCpOptions(DistCpOptions that)
that
- DistCpOptions being copied from.public boolean shouldAtomicCommit()
public void setAtomicCommit(boolean atomicCommit)
atomicCommit
- - boolean switchpublic boolean shouldSyncFolder()
public void setSyncFolder(boolean syncFolder)
syncFolder
- - boolean switchpublic boolean shouldDeleteMissing()
public void setDeleteMissing(boolean deleteMissing)
deleteMissing
- - boolean switchpublic boolean shouldIgnoreFailures()
public void setIgnoreFailures(boolean ignoreFailures)
ignoreFailures
- - boolean switchpublic boolean shouldBlock()
public void setBlocking(boolean blocking)
blocking
- - boolean switchpublic boolean shouldOverwrite()
public void setOverwrite(boolean overwrite)
overwrite
- - boolean switchpublic boolean shouldSkipCRC()
public void setSkipCRC(boolean skipCRC)
skipCRC
- - boolean switchpublic int getMaxMaps()
public void setMaxMaps(int maxMaps)
maxMaps
- - Number of mapspublic int getMapBandwidth()
public void setMapBandwidth(int mapBandwidth)
mapBandwidth
- - per map bandwidthpublic String getSslConfigurationFile()
public void setSslConfigurationFile(String sslConfigurationFile)
sslConfigurationFile
- - Local ssl config file pathpublic Iterator<DistCpOptions.FileAttribute> preserveAttributes()
public boolean shouldPreserve(DistCpOptions.FileAttribute attribute)
attribute
- - Attribute to checkpublic void preserve(DistCpOptions.FileAttribute fileAttribute)
fileAttribute
- - Attribute to add, one at a timepublic Path getAtomicWorkPath()
public void setAtomicWorkPath(Path atomicWorkPath)
atomicWorkPath
- - Path on the target clusterpublic Path getLogPath()
public void setLogPath(Path logPath)
logPath
- - Path where logs will be savedpublic String getCopyStrategy()
public void setCopyStrategy(String copyStrategy)
copyStrategy
- - copy Strategy to usepublic Path getSourceFileListing()
public void setSourcePaths(List<Path> sourcePaths)
sourcePaths
- The new list of source-paths.public Path getTargetPath()
public void validate(DistCpOptionSwitch option, boolean value)
public void appendToConf(Configuration conf)
conf
- - Configruation object to which the options need to be addedpublic String toString()
protected DistCpOptions clone() throws CloneNotSupportedException
clone
在类中 Object
CloneNotSupportedException
Copyright © 2009 The Apache Software Foundation