org.apache.hadoop.hdfs.server.datanode

接口 FSDatasetInterface

所有超级接口:

FSDatasetMBean

所有已知实现类:

FSDataset

public interface FSDatasetInterface
extends FSDatasetMBean

This is an interface for the underlying storage that stores blocks for a data node. Examples are the FSDataset (which stores blocks on dirs) and SimulatedFSDataset (which simulates data).

嵌套类概要

嵌套类

限定符和类型	接口和说明
static class	FSDatasetInterface.BlockInputStreams This class contains the input streams for the data and checksum of a block
static class	FSDatasetInterface.BlockWriteStreams This class contains the output streams for the data and checksum of a block
static class	FSDatasetInterface.MetaDataInputStream This class provides the input stream and length of the metadata of a block

方法概要

方法

限定符和类型	方法和说明
void	checkDataDir() Check if all the data directories are healthy
void	finalizeBlock(Block b) Finalizes the block previously opened for writing using writeToBlock.
void	finalizeBlockIfNeeded(Block b) Finalizes the block previously opened for writing using writeToBlock if not already finalized
InputStream	getBlockInputStream(Block b) Returns an input stream to read the contents of the specified block
InputStream	getBlockInputStream(Block b, long seekOffset) Returns an input stream at specified offset of the specified block
BlockLocalPathInfo	getBlockLocalPathInfo(Block b) Get BlockLocalPathInfo for the given block.
Block[]	getBlockReport() Returns the block report - the full list of blocks stored Returns only finalized blocks
Block[]	getBlocksBeingWrittenReport() Returns the blocks being written report
long	getChannelPosition(Block b, FSDatasetInterface.BlockWriteStreams stream) Returns the current offset in the data stream.
long	getLength(Block b) Returns the specified block's on-disk length (excluding metadata)
FSDatasetInterface.MetaDataInputStream	getMetaDataInputStream(Block b) Returns metaData of block b as an input stream (and its length)
long	getMetaDataLength(Block b) Returns the length of the metadata file of the specified block
Block	getStoredBlock(long blkid)
FSDatasetInterface.BlockInputStreams	getTmpInputStreams(Block b, long blkoff, long ckoff) Returns an input stream at specified offset of the specified block The block is still in the tmp directory and is not finalized
long	getVisibleLength(Block b) Returns the specified block's visible length (has metadata for this)
boolean	hasEnoughResource() checks how many valid storage volumes are there in the DataNode
void	invalidate(Block[] invalidBlks) Invalidates the specified blocks
boolean	isAsyncBlockReportReady()
boolean	isValidBlock(Block b) Is the block valid?
boolean	metaFileExists(Block b) Does the meta file exist for this block?
void	requestAsyncBlockReport() Request that a block report be prepared.
Block[]	retrieveAsyncBlockReport() Retrieve an asynchronously prepared block report.
void	setChannelPosition(Block b, FSDatasetInterface.BlockWriteStreams stream, long dataOffset, long ckOffset) Sets the file pointer of the data stream and checksum stream to the specified values.
void	setVisibleLength(Block b, long length) update the specified blocks visible meta data.
void	shutdown() Shutdown the FSDataset
BlockRecoveryInfo	startBlockRecovery(long blockId)
String	toString() Stringifies the name of the storage
void	unfinalizeBlock(Block b) Unfinalizes the block previously opened for writing using writeToBlock.
void	updateBlock(Block oldblock, Block newblock) Update the block to the new generation stamp and length.
void	validateBlockMetadata(Block b) Validate that the contents in the Block matches the file on disk.
FSDatasetInterface.BlockWriteStreams	writeToBlock(Block b, boolean isRecovery, boolean isReplicationRequest) Creates the block and returns output streams to write data and CRC

从接口继承的方法 org.apache.hadoop.hdfs.server.datanode.metrics.FSDatasetMBean

getCapacity, getDfsUsed, getRemaining, getStorageInfo

方法详细资料

getMetaDataLength

long getMetaDataLength(Block b)
                       throws IOException

Returns the length of the metadata file of the specified block

参数:

b - - the block for which the metadata length is desired

返回:

the length of the metadata file for the specified block.

抛出:

IOException

getMetaDataInputStream

FSDatasetInterface.MetaDataInputStream getMetaDataInputStream(Block b)
                                                              throws IOException

Returns metaData of block b as an input stream (and its length)

参数:

b - - the block

返回:

the metadata input stream;

抛出:

IOException

metaFileExists

boolean metaFileExists(Block b)
                       throws IOException

Does the meta file exist for this block?

参数:

b - - the block

返回:

true of the metafile for specified block exits

抛出:

IOException

getLength

long getLength(Block b)
               throws IOException

Returns the specified block's on-disk length (excluding metadata)

参数:

b -

返回:

the specified block's on-disk length (excluding metadta)

抛出:

IOException

getVisibleLength

long getVisibleLength(Block b)
                      throws IOException

Returns the specified block's visible length (has metadata for this)

参数:

b -

返回:

the specified block's visible length

抛出:

IOException

setVisibleLength

void setVisibleLength(Block b,
                    long length)
                      throws IOException

update the specified blocks visible meta data. NOTE: only applies to blocks that are being written to. If called on closed blocks, throws IOException

参数:

b - block to update the length for

length - value to set visible length to

抛出:

IOException - if the block is not in ongoingCreates

getStoredBlock

Block getStoredBlock(long blkid)
                     throws IOException

返回:

the generation stamp stored with the block.

抛出:

IOException

getBlockInputStream

InputStream getBlockInputStream(Block b)
                                throws IOException

Returns an input stream to read the contents of the specified block

参数:

b -

返回:

an input stream to read the contents of the specified block

抛出:

IOException

getBlockInputStream

InputStream getBlockInputStream(Block b,
                              long seekOffset)
                                throws IOException

Returns an input stream at specified offset of the specified block

参数:

b -

seekOffset -

返回:

an input stream to read the contents of the specified block, starting at the offset

抛出:

IOException

getTmpInputStreams

FSDatasetInterface.BlockInputStreams getTmpInputStreams(Block b,
                                                      long blkoff,
                                                      long ckoff)
                                                        throws IOException

Returns an input stream at specified offset of the specified block The block is still in the tmp directory and is not finalized

参数:

b -

blkoff -

ckoff -

返回:

an input stream to read the contents of the specified block, starting at the offset

抛出:

IOException

writeToBlock

FSDatasetInterface.BlockWriteStreams writeToBlock(Block b,
                                                boolean isRecovery,
                                                boolean isReplicationRequest)
                                                  throws IOException

Creates the block and returns output streams to write data and CRC

参数:

b -

isRecovery - True if this is part of error recovery, otherwise false

isReplicationRequest - True if this is part of block replication request

返回:

a BlockWriteStreams object to allow writing the block data and CRC

抛出:

IOException

updateBlock

void updateBlock(Block oldblock,
               Block newblock)
                 throws IOException

Update the block to the new generation stamp and length.

抛出:

IOException

finalizeBlock

void finalizeBlock(Block b)
                   throws IOException

Finalizes the block previously opened for writing using writeToBlock. The block size is what is in the parameter b and it must match the amount of data written

参数:

b -

抛出:

IOException

finalizeBlockIfNeeded

void finalizeBlockIfNeeded(Block b)
                           throws IOException

Finalizes the block previously opened for writing using writeToBlock if not already finalized

参数:

b -

抛出:

IOException

unfinalizeBlock

void unfinalizeBlock(Block b)
                     throws IOException

Unfinalizes the block previously opened for writing using writeToBlock. The temporary file associated with this block is deleted.

参数:

b -

抛出:

IOException

getBlockReport

Block[] getBlockReport()

Returns the block report - the full list of blocks stored Returns only finalized blocks

返回:

- the block report - the full list of blocks stored

requestAsyncBlockReport

void requestAsyncBlockReport()

Request that a block report be prepared.

isAsyncBlockReportReady

boolean isAsyncBlockReportReady()

返回:

true if an asynchronous block report is ready

retrieveAsyncBlockReport

Block[] retrieveAsyncBlockReport()

Retrieve an asynchronously prepared block report. Callers should first call requestAsyncBlockReport(), and then poll isAsyncBlockReportReady() until it returns true. Retrieving the asynchronous block report also resets it; a new one must be prepared before this method may be called again.

抛出:

IllegalStateException - if an async report is not ready

getBlocksBeingWrittenReport

Block[] getBlocksBeingWrittenReport()

Returns the blocks being written report

返回:

- the blocks being written report

isValidBlock

boolean isValidBlock(Block b)

Is the block valid?

参数:

b -

返回:

- true if the specified block is valid

invalidate

void invalidate(Block[] invalidBlks)
                throws IOException

Invalidates the specified blocks

参数:

invalidBlks - - the blocks to be invalidated

抛出:

IOException

checkDataDir

void checkDataDir()
                  throws DiskChecker.DiskErrorException

Check if all the data directories are healthy

抛出:

DiskChecker.DiskErrorException

toString

String toString()

Stringifies the name of the storage

覆盖:

toString 在类中 Object

shutdown

void shutdown()

Shutdown the FSDataset

getChannelPosition

long getChannelPosition(Block b,
                      FSDatasetInterface.BlockWriteStreams stream)
                        throws IOException

Returns the current offset in the data stream.

参数:

b -

stream - The stream to the data file and checksum file

返回:

the position of the file pointer in the data stream

抛出:

IOException

setChannelPosition

void setChannelPosition(Block b,
                      FSDatasetInterface.BlockWriteStreams stream,
                      long dataOffset,
                      long ckOffset)
                        throws IOException

Sets the file pointer of the data stream and checksum stream to the specified values.

参数:

b -

stream - The stream for the data file and checksum file

dataOffset - The position to which the file pointre for the data stream should be set

ckOffset - The position to which the file pointre for the checksum stream should be set

抛出:

IOException

validateBlockMetadata

void validateBlockMetadata(Block b)
                           throws IOException

Validate that the contents in the Block matches the file on disk. Returns true if everything is fine.

参数:

b - The block to be verified.

抛出:

IOException

hasEnoughResource

boolean hasEnoughResource()

checks how many valid storage volumes are there in the DataNode

返回:

true if more then minimum valid volumes left in the FSDataSet

startBlockRecovery

BlockRecoveryInfo startBlockRecovery(long blockId)
                                     throws IOException

抛出:

IOException

getBlockLocalPathInfo

BlockLocalPathInfo getBlockLocalPathInfo(Block b)
                                         throws IOException

Get BlockLocalPathInfo for the given block.

抛出:

IOException