org.apache.hadoop.mapreduce.lib.partition

类 TotalOrderPartitioner<K extends WritableComparable<?>,V>

java.lang.Object

org.apache.hadoop.mapreduce.Partitioner<K,V>

org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner<K,V>

所有已实现的接口:

Configurable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class TotalOrderPartitioner<K extends WritableComparable<?>,V>
extends Partitioner<K,V>
implements Configurable

Partitioner effecting a total order by reading split points from an externally generated source.

字段概要

字段

限定符和类型	字段和说明
static String	DEFAULT_PATH
static String	MAX_TRIE_DEPTH
static String	NATURAL_ORDER
static String	PARTITIONER_PATH

构造器概要

构造器

构造器和说明
TotalOrderPartitioner()

方法概要

方法

限定符和类型	方法和说明
Configuration	getConf() Return the configuration used by this object.
int	getPartition(K key, V value, int numPartitions) Get the partition number for a given key (hence record) given the total number of partitions i.e. number of reduce-tasks for the job.
static String	getPartitionFile(Configuration conf) Get the path to the SequenceFile storing the sorted partition keyset.
void	setConf(Configuration conf) Read in the partition file and build indexing data structures.
static void	setPartitionFile(Configuration conf, Path p) Set the path to the SequenceFile storing the sorted partition keyset.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

DEFAULT_PATH

public static final String DEFAULT_PATH

另请参阅:

常量字段值

PARTITIONER_PATH

public static final String PARTITIONER_PATH

另请参阅:

常量字段值

MAX_TRIE_DEPTH

public static final String MAX_TRIE_DEPTH

另请参阅:

常量字段值

NATURAL_ORDER

public static final String NATURAL_ORDER

另请参阅:

常量字段值

构造器详细资料

TotalOrderPartitioner

public TotalOrderPartitioner()

方法详细资料

setConf

public void setConf(Configuration conf)

Read in the partition file and build indexing data structures. If the keytype is BinaryComparable and total.order.partitioner.natural.order is not false, a trie of the first total.order.partitioner.max.trie.depth(2) + 1 bytes will be built. Otherwise, keys will be located using a binary search of the partition keyset using the RawComparator defined for this job. The input file must be sorted with the same comparator and contain JobContext.getNumReduceTasks() - 1 keys.

指定者:

setConf 在接口中 Configurable

getConf

public Configuration getConf()

从接口复制的说明: Configurable

Return the configuration used by this object.

指定者:

getConf 在接口中 Configurable

getPartition

public int getPartition(K key,
               V value,
               int numPartitions)

从类复制的说明: Partitioner

Get the partition number for a given key (hence record) given the total number of partitions i.e. number of reduce-tasks for the job.

Typically a hash function on a all or a subset of the key.

指定者:

getPartition 在类中 Partitioner<K extends WritableComparable<?>,V>

参数:

key - the key to be partioned.

value - the entry value.

numPartitions - the total number of partitions.

返回:

the partition number for the key.

setPartitionFile

public static void setPartitionFile(Configuration conf,
                    Path p)

Set the path to the SequenceFile storing the sorted partition keyset. It must be the case that for R reduces, there are R-1 keys in the SequenceFile.

getPartitionFile

public static String getPartitionFile(Configuration conf)

Get the path to the SequenceFile storing the sorted partition keyset.

另请参阅:

setPartitionFile(Configuration, Path)