org.apache.hadoop.hbase.io.hfile

Class AbstractHFileReader

java.lang.Object

org.apache.hadoop.hbase.io.hfile.AbstractHFileReader

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, HFile.CachingBlockReader, HFile.Reader

Direct Known Subclasses:

HFileReaderV2

@InterfaceAudience.Private
public abstract class AbstractHFileReader
extends Object
implements HFile.Reader, org.apache.hadoop.conf.Configurable

Common functionality needed by all versions of HFile readers.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbstractHFileReader.BlockIndexNotLoadedException
static class	AbstractHFileReader.NotSeekedException An exception thrown when an operation requiring a scanner to be seeked is invoked on a scanner that is not seeked.
protected static class	AbstractHFileReader.Scanner

Field Summary

Fields

Modifier and Type	Field and Description
protected int	avgKeyLen Average key length read from file info
protected int	avgValueLen Average value length read from file info
protected CacheConfig	cacheConf Block cache configuration.
protected KeyValue.KVComparator	comparator Key comparator
protected Compression.Algorithm	compressAlgo Filled when we read in the trailer.
protected org.apache.hadoop.conf.Configuration	conf
protected HFileDataBlockEncoder	dataBlockEncoder What kind of data block encoding should be used while reading, writing, and handling cache.
protected HFileBlockIndex.BlockIndexReader	dataBlockIndexReader Data block index reader keeping the root data index in memory
protected HFile.FileInfo	fileInfo
protected long	fileSize Size of this file.
protected HFileSystem	hfs The filesystem used for accesing data
protected org.apache.hadoop.fs.FSDataInputStream	istream Stream to read from.
protected org.apache.hadoop.fs.FSDataInputStream	istreamNoFsChecksum The file system stream of the underlying HFile that does not do checksum verification in the file system
protected byte[]	lastKey Last key in the file.
protected HFileBlockIndex.BlockIndexReader	metaBlockIndexReader Meta block index reader -- always single level
protected String	name File name to be used for block names
protected org.apache.hadoop.fs.Path	path Path of file
protected FixedFileTrailer	trailer

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractHFileReader(org.apache.hadoop.fs.Path path, FixedFileTrailer trailer, long fileSize, CacheConfig cacheConf, HFileSystem hfs, org.apache.hadoop.conf.Configuration conf)

Method Summary

Methods

Modifier and Type	Method and Description
KeyValue.KVComparator	getComparator()
Compression.Algorithm	getCompressionAlgorithm()
org.apache.hadoop.conf.Configuration	getConf()
DataBlockEncoding	getDataBlockEncoding()
HFileBlockIndex.BlockIndexReader	getDataBlockIndexReader()
long	getEntries()
byte[]	getFirstKey()
byte[]	getFirstRowKey() TODO left from HFile version 1: move this to StoreFile after Ryan's patch goes in to eliminate KeyValue here.
byte[]	getLastRowKey() TODO left from HFile version 1: move this to StoreFile after Ryan's patch goes in to eliminate KeyValue here.
abstract int	getMajorVersion()
String	getName() Returns this reader's "name".
org.apache.hadoop.fs.Path	getPath()
HFileScanner	getScanner(boolean cacheBlocks, boolean pread) Create a Scanner on this file.
FixedFileTrailer	getTrailer()
long	indexSize()
abstract boolean	isFileInfoLoaded()
long	length()
HFile.FileInfo	loadFileInfo()
void	setConf(org.apache.hadoop.conf.Configuration conf)
String	toString()
protected String	toStringFirstKey()
protected String	toStringLastKey()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.io.hfile.HFile.Reader

close, getDeleteBloomFilterMetadata, getFileContext, getGeneralBloomFilterMetadata, getLastKey, getMetaBlock, getScanner, hasMVCCInfo, midkey

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface org.apache.hadoop.hbase.io.hfile.HFile.CachingBlockReader

readBlock

Field Detail

istream

protected org.apache.hadoop.fs.FSDataInputStream istream

Stream to read from. Does checksum verifications in file system

istreamNoFsChecksum

protected org.apache.hadoop.fs.FSDataInputStream istreamNoFsChecksum

The file system stream of the underlying HFile that does not do checksum verification in the file system

dataBlockIndexReader

protected HFileBlockIndex.BlockIndexReader dataBlockIndexReader

Data block index reader keeping the root data index in memory

metaBlockIndexReader

protected HFileBlockIndex.BlockIndexReader metaBlockIndexReader

Meta block index reader -- always single level

trailer

protected final FixedFileTrailer trailer

compressAlgo

protected final Compression.Algorithm compressAlgo

Filled when we read in the trailer.

dataBlockEncoder

protected HFileDataBlockEncoder dataBlockEncoder

What kind of data block encoding should be used while reading, writing, and handling cache.

lastKey

protected byte[] lastKey

Last key in the file. Filled in when we read in the file info

avgKeyLen

protected int avgKeyLen

Average key length read from file info

avgValueLen

protected int avgValueLen

Average value length read from file info

comparator

protected KeyValue.KVComparator comparator

Key comparator

fileSize

protected final long fileSize

Size of this file.

cacheConf

protected final CacheConfig cacheConf

Block cache configuration.

path

protected final org.apache.hadoop.fs.Path path

Path of file

name

protected final String name

File name to be used for block names

fileInfo

protected HFile.FileInfo fileInfo

hfs

protected HFileSystem hfs

The filesystem used for accesing data

conf

protected org.apache.hadoop.conf.Configuration conf

Constructor Detail

AbstractHFileReader

protected AbstractHFileReader(org.apache.hadoop.fs.Path path,
                   FixedFileTrailer trailer,
                   long fileSize,
                   CacheConfig cacheConf,
                   HFileSystem hfs,
                   org.apache.hadoop.conf.Configuration conf)

Method Detail

toStringFirstKey

protected String toStringFirstKey()

toStringLastKey

protected String toStringLastKey()

isFileInfoLoaded

public abstract boolean isFileInfoLoaded()

toString

public String toString()

Overrides:

toString in class Object

length

public long length()

Specified by:

length in interface HFile.Reader

getScanner

public HFileScanner getScanner(boolean cacheBlocks,
                      boolean pread)

Create a Scanner on this file. No seeks or reads are done on creation. Call HFileScanner.seekTo(byte[]) to position an start the read. There is nothing to clean up in a Scanner. Letting go of your references to the scanner is sufficient. NOTE: Do not use this overload of getScanner for compactions.

Specified by:

getScanner in interface HFile.Reader

Parameters:

cacheBlocks - True if we should cache blocks read in by this scanner.

pread - Use positional read rather than seek+read if true (pread is better for random reads, seek+read is better scanning).

Returns:

Scanner on this file.

getFirstKey

public byte[] getFirstKey()

Specified by:

getFirstKey in interface HFile.Reader

Returns:

the first key in the file. May be null if file has no entries. Note that this is not the first row key, but rather the byte form of the first KeyValue.

getFirstRowKey

public byte[] getFirstRowKey()

TODO left from HFile version 1: move this to StoreFile after Ryan's patch goes in to eliminate KeyValue here.

Specified by:

getFirstRowKey in interface HFile.Reader

Returns:

the first row key, or null if the file is empty.

getLastRowKey

public byte[] getLastRowKey()

TODO left from HFile version 1: move this to StoreFile after Ryan's patch goes in to eliminate KeyValue here.

Specified by:

getLastRowKey in interface HFile.Reader

Returns:

the last row key, or null if the file is empty.

getEntries

public long getEntries()

Specified by:

getEntries in interface HFile.Reader

Returns:

number of KV entries in this HFile

getComparator

public KeyValue.KVComparator getComparator()

Specified by:

getComparator in interface HFile.Reader

Returns:

comparator

getCompressionAlgorithm

public Compression.Algorithm getCompressionAlgorithm()

Specified by:

getCompressionAlgorithm in interface HFile.Reader

Returns:

compression algorithm

indexSize

public long indexSize()

Specified by:

indexSize in interface HFile.Reader

Returns:

the total heap size of data and meta block indexes in bytes. Does not take into account non-root blocks of a multilevel data index.

getName

public String getName()

Description copied from interface: HFile.Reader

Returns this reader's "name". Usually the last component of the path. Needs to be constant as the file is being moved to support caching on write.

Specified by:

getName in interface HFile.Reader

getDataBlockIndexReader

public HFileBlockIndex.BlockIndexReader getDataBlockIndexReader()

Specified by:

getDataBlockIndexReader in interface HFile.Reader

getTrailer

public FixedFileTrailer getTrailer()

Specified by:

getTrailer in interface HFile.Reader

loadFileInfo

public HFile.FileInfo loadFileInfo()
                            throws IOException

Specified by:

loadFileInfo in interface HFile.Reader

Throws:

IOException

getPath

public org.apache.hadoop.fs.Path getPath()

Specified by:

getPath in interface HFile.Reader

getDataBlockEncoding

public DataBlockEncoding getDataBlockEncoding()

Specified by:

getDataBlockEncoding in interface HFile.Reader

getMajorVersion

public abstract int getMajorVersion()

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable