org.apache.hadoop.mapreduce.lib.partition

类 InputSampler<K,V>

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.mapreduce.lib.partition.InputSampler<K,V>

所有已实现的接口:

Configurable, Tool

@InterfaceAudience.Public
@InterfaceStability.Stable
public class InputSampler<K,V>
extends Configured
implements Tool

Utility for collecting samples and writing a partition file for TotalOrderPartitioner.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	InputSampler.IntervalSampler<K,V> Sample from s splits at regular intervals.
static class	InputSampler.RandomSampler<K,V> Sample from random points in the input.
static interface	InputSampler.Sampler<K,V> Interface to sample using an InputFormat.
static class	InputSampler.SplitSampler<K,V> Samples the first n records from s splits.

构造器概要

构造器

构造器和说明
InputSampler(Configuration conf)

方法概要

方法

限定符和类型	方法和说明
static void	main(String[] args)
int	run(String[] args) Driver for InputSampler from the command line.
static <K,V> void	writePartitionFile(Job job, InputSampler.Sampler<K,V> sampler) Write a partition file for the given job, using the Sampler provided.

从类继承的方法 org.apache.hadoop.conf.Configured

getConf, setConf

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.hadoop.conf.Configurable

getConf, setConf

构造器详细资料

InputSampler

public InputSampler(Configuration conf)

方法详细资料

writePartitionFile

public static <K,V> void writePartitionFile(Job job,
                            InputSampler.Sampler<K,V> sampler)
                               throws IOException,
                                      ClassNotFoundException,
                                      InterruptedException

Write a partition file for the given job, using the Sampler provided. Queries the sampler for a sample keyset, sorts by the output key comparator, selects the keys for each rank, and writes to the destination returned from TotalOrderPartitioner.getPartitionFile(org.apache.hadoop.conf.Configuration).

抛出:

IOException

ClassNotFoundException

InterruptedException

run

public int run(String[] args)
        throws Exception

Driver for InputSampler from the command line. Configures a JobConf instance and calls writePartitionFile(org.apache.hadoop.mapreduce.Job, org.apache.hadoop.mapreduce.lib.partition.InputSampler.Sampler<K, V>).

指定者:

run 在接口中 Tool

参数:

args - command specific arguments.

返回:

exit code.

抛出:

Exception

main

public static void main(String[] args)
                 throws Exception

抛出:

Exception