org.apache.hadoop.mapreduce.lib.partition

类 BinaryPartitioner<V>

java.lang.Object

org.apache.hadoop.mapreduce.Partitioner<BinaryComparable,V>

org.apache.hadoop.mapreduce.lib.partition.BinaryPartitioner<V>

所有已实现的接口:

Configurable

直接已知子类:

BinaryPartitioner

@InterfaceAudience.Public
@InterfaceStability.Evolving
public class BinaryPartitioner<V>
extends Partitioner<BinaryComparable,V>
implements Configurable

Partition BinaryComparable keys using a configurable part of the bytes array returned by BinaryComparable.getBytes().

The subarray to be used for the partitioning can be defined by means of the following properties:

• mapred.binary.partitioner.left.offset: left offset in array (0 by default)
• mapred.binary.partitioner.right.offset: right offset in array (-1 by default)

Like in Python, both negative and positive offsets are allowed, but the meaning is slightly different. In case of an array of length 5, for instance, the possible offsets are:

  +---+---+---+---+---+
  | B | B | B | B | B |
  +---+---+---+---+---+
    0   1   2   3   4
   -5  -4  -3  -2  -1
 

The first row of numbers gives the position of the offsets 0...5 in the array; the second row gives the corresponding negative offsets. Contrary to Python, the specified subarray has byte i and j as first and last element, repectively, when i and j are the left and right offset.

For Hadoop programs written in Java, it is advisable to use one of the following static convenience methods for setting the offsets:

• setOffsets(org.apache.hadoop.conf.Configuration, int, int)
• setLeftOffset(org.apache.hadoop.conf.Configuration, int)
• setRightOffset(org.apache.hadoop.conf.Configuration, int)

构造器概要

构造器

构造器和说明
BinaryPartitioner()

方法概要

方法

限定符和类型	方法和说明
Configuration	getConf() Return the configuration used by this object.
int	getPartition(BinaryComparable key, V value, int numPartitions) Use (the specified slice of the array returned by) BinaryComparable.getBytes() to partition.
void	setConf(Configuration conf) Set the configuration to be used by this object.
static void	setLeftOffset(Configuration conf, int offset) Set the subarray to be used for partitioning to bytes[offset:] in Python syntax.
static void	setOffsets(Configuration conf, int left, int right) Set the subarray to be used for partitioning to bytes[left:(right+1)] in Python syntax.
static void	setRightOffset(Configuration conf, int offset) Set the subarray to be used for partitioning to bytes[:(offset+1)] in Python syntax.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

BinaryPartitioner

public BinaryPartitioner()

方法详细资料

setOffsets

public static void setOffsets(Configuration conf,
              int left,
              int right)

Set the subarray to be used for partitioning to bytes[left:(right+1)] in Python syntax.

参数:

conf - configuration object

left - left Python-style offset

right - right Python-style offset

setLeftOffset

public static void setLeftOffset(Configuration conf,
                 int offset)

Set the subarray to be used for partitioning to bytes[offset:] in Python syntax.

参数:

conf - configuration object

offset - left Python-style offset

setRightOffset

public static void setRightOffset(Configuration conf,
                  int offset)

Set the subarray to be used for partitioning to bytes[:(offset+1)] in Python syntax.

参数:

conf - configuration object

offset - right Python-style offset

setConf

public void setConf(Configuration conf)

从接口复制的说明: Configurable

Set the configuration to be used by this object.

指定者:

setConf 在接口中 Configurable

getConf

public Configuration getConf()

从接口复制的说明: Configurable

Return the configuration used by this object.

指定者:

getConf 在接口中 Configurable

getPartition

public int getPartition(BinaryComparable key,
               V value,
               int numPartitions)

Use (the specified slice of the array returned by) BinaryComparable.getBytes() to partition.

指定者:

getPartition 在类中 Partitioner<BinaryComparable,V>

参数:

key - the key to be partioned.

value - the entry value.

numPartitions - the total number of partitions.

返回:

the partition number for the key.