org.apache.hadoop.fs

类 FSInputChecker

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

org.apache.hadoop.fs.FSInputChecker

所有已实现的接口:

Closeable, AutoCloseable, PositionedReadable, Seekable

直接已知子类:

DFSClient.RemoteBlockReader

public abstract class FSInputChecker
extends FSInputStream

This is a generic input stream for verifying checksums for data before it is read by a user.

字段概要

字段

限定符和类型	字段和说明
protected Path	file The file name from which data is read from
static org.apache.commons.logging.Log	LOG

构造器概要

构造器

限定符	构造器和说明
protected	FSInputChecker(Path file, int numOfRetries) Constructor
protected	FSInputChecker(Path file, int numOfRetries, boolean verifyChecksum, Checksum sum, int chunkSize, int checksumSize) Constructor

方法概要

方法

限定符和类型	方法和说明
int	available()
static long	checksum2long(byte[] checksum) Convert a checksum byte array to a long
protected abstract long	getChunkPosition(long pos) Return position of beginning of chunk containing pos.
long	getPos() Return the current offset from the start of the file
void	mark(int readlimit)
boolean	markSupported()
protected boolean	needChecksum() Return true if there is a need for checksum verification
int	read() Read one checksum-verified byte
int	read(byte[] b, int off, int len) Read checksum verified bytes from this byte-input stream into the specified byte array, starting at the given offset.
protected abstract int	readChunk(long pos, byte[] buf, int offset, int len, byte[] checksum) Reads in next checksum chunk data into buf at offset and checksum into checksum.
protected static int	readFully(InputStream stm, byte[] buf, int offset, int len) A utility function that tries to read up to len bytes from stm
void	reset()
void	seek(long pos) Seek to the given position in the stream.
protected void	set(boolean verifyChecksum, Checksum sum, int maxChunkSize, int checksumSize) Set the checksum related parameters
long	skip(long n) Skips over and discards n bytes of data from the input stream.

从类继承的方法 org.apache.hadoop.fs.FSInputStream

read, readFully, readFully, seekToNewSource

从类继承的方法 java.io.InputStream

close, read

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

public static final org.apache.commons.logging.Log LOG

file

protected Path file

The file name from which data is read from

构造器详细资料

FSInputChecker

protected FSInputChecker(Path file,
              int numOfRetries)

Constructor

参数:

file - The name of the file to be read

numOfRetries - Number of read retries when ChecksumError occurs

FSInputChecker

protected FSInputChecker(Path file,
              int numOfRetries,
              boolean verifyChecksum,
              Checksum sum,
              int chunkSize,
              int checksumSize)

Constructor

参数:

file - The name of the file to be read

numOfRetries - Number of read retries when ChecksumError occurs

sum - the type of Checksum engine

chunkSize - maximun chunk size

checksumSize - the number byte of each checksum

方法详细资料

readChunk

protected abstract int readChunk(long pos,
            byte[] buf,
            int offset,
            int len,
            byte[] checksum)
                          throws IOException

Reads in next checksum chunk data into buf at offset and checksum into checksum. The method is used for implementing read, therefore, it should be optimized for sequential reading

参数:

pos - chunkPos

buf - desitination buffer

offset - offset in buf at which to store data

len - maximun number of bytes to read

返回:

number of bytes read

抛出:

IOException

getChunkPosition

protected abstract long getChunkPosition(long pos)

Return position of beginning of chunk containing pos.

参数:

pos - a postion in the file

返回:

the starting position of the chunk which contains the byte

needChecksum

protected boolean needChecksum()

Return true if there is a need for checksum verification

read

public int read()
         throws IOException

Read one checksum-verified byte

指定者:

read 在类中 InputStream

返回:

the next byte of data, or -1 if the end of the stream is reached.

抛出:

IOException - if an I/O error occurs.

read

public int read(byte[] b,
       int off,
       int len)
         throws IOException

Read checksum verified bytes from this byte-input stream into the specified byte array, starting at the given offset.

This method implements the general contract of the corresponding read method of the InputStream class. As an additional convenience, it attempts to read as many bytes as possible by repeatedly invoking the read method of the underlying stream. This iterated read continues until one of the following conditions becomes true:

• The specified number of bytes have been read,
• The read method of the underlying stream returns -1, indicating end-of-file.

If the first read on the underlying stream returns -1 to indicate end-of-file then this method returns -1. Otherwise this method returns the number of bytes actually read.

覆盖:

read 在类中 InputStream

参数:

b - destination buffer.

off - offset at which to start storing bytes.

len - maximum number of bytes to read.

返回:

the number of bytes read, or -1 if the end of the stream has been reached.

抛出:

IOException - if an I/O error occurs. ChecksumException if any checksum error occurs

checksum2long

public static long checksum2long(byte[] checksum)

Convert a checksum byte array to a long

getPos

public long getPos()
            throws IOException

从类复制的说明: FSInputStream

Return the current offset from the start of the file

指定者:

getPos 在接口中 Seekable

指定者:

getPos 在类中 FSInputStream

抛出:

IOException

available

public int available()
              throws IOException

覆盖:

available 在类中 InputStream

抛出:

IOException

skip

public long skip(long n)
          throws IOException

Skips over and discards n bytes of data from the input stream.

This method may skip more bytes than are remaining in the backing file. This produces no exception and the number of bytes skipped may include some number of bytes that were beyond the EOF of the backing file. Attempting to read from the stream after skipping past the end will result in -1 indicating the end of the file.

If n is negative, no bytes are skipped.

覆盖:

skip 在类中 InputStream

参数:

n - the number of bytes to be skipped.

返回:

the actual number of bytes skipped.

抛出:

IOException - if an I/O error occurs. ChecksumException if the chunk to skip to is corrupted

seek

public void seek(long pos)
          throws IOException

Seek to the given position in the stream. The next read() will be from that position.

This method may seek past the end of the file. This produces no exception and an attempt to read from the stream will result in -1 indicating the end of the file.

指定者:

seek 在接口中 Seekable

指定者:

seek 在类中 FSInputStream

参数:

pos - the postion to seek to.

抛出:

IOException - if an I/O error occurs. ChecksumException if the chunk to seek to is corrupted

readFully

protected static int readFully(InputStream stm,
            byte[] buf,
            int offset,
            int len)
                        throws IOException

A utility function that tries to read up to len bytes from stm

参数:

stm - an input stream

buf - destiniation buffer

offset - offset at which to store data

len - number of bytes to read

返回:

actual number of bytes read

抛出:

IOException - if there is any IO error

set

protected final void set(boolean verifyChecksum,
       Checksum sum,
       int maxChunkSize,
       int checksumSize)

Set the checksum related parameters

参数:

verifyChecksum - whether to verify checksum

sum - which type of checksum to use

maxChunkSize - maximun chunk size

checksumSize - checksum size

markSupported

public final boolean markSupported()

覆盖:

markSupported 在类中 InputStream

mark

public final void mark(int readlimit)

覆盖:

mark 在类中 InputStream

reset

public final void reset()
                 throws IOException

覆盖:

reset 在类中 InputStream

抛出:

IOException