org.apache.hadoop.mapred.lib

类 InputSampler.RandomSampler<K,V>

java.lang.Object

org.apache.hadoop.mapred.lib.InputSampler.RandomSampler<K,V>

所有已实现的接口:

InputSampler.Sampler<K,V>

封闭类:

InputSampler<K,V>

public static class InputSampler.RandomSampler<K,V>
extends Object
implements InputSampler.Sampler<K,V>

Sample from random points in the input. General-purpose sampler. Takes numSamples / maxSplitsSampled inputs from each split.

构造器概要

构造器

构造器和说明
InputSampler.RandomSampler(double freq, int numSamples) Create a new RandomSampler sampling all splits.
InputSampler.RandomSampler(double freq, int numSamples, int maxSplitsSampled) Create a new RandomSampler.

方法概要

方法

限定符和类型	方法和说明
K[]	getSample(InputFormat<K,V> inf, JobConf job) Randomize the split order, then take the specified number of keys from each split sampled, where each key is selected with the specified probability and possibly replaced by a subsequently selected key when the quota of keys from that split is satisfied.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

InputSampler.RandomSampler

public InputSampler.RandomSampler(double freq,
                          int numSamples)

Create a new RandomSampler sampling all splits. This will read every split at the client, which is very expensive.

参数:

freq - Probability with which a key will be chosen.

numSamples - Total number of samples to obtain from all selected splits.

InputSampler.RandomSampler

public InputSampler.RandomSampler(double freq,
                          int numSamples,
                          int maxSplitsSampled)

Create a new RandomSampler.

参数:

freq - Probability with which a key will be chosen.

numSamples - Total number of samples to obtain from all selected splits.

maxSplitsSampled - The maximum number of splits to examine.

方法详细资料

getSample

public K[] getSample(InputFormat<K,V> inf,
            JobConf job)
              throws IOException

Randomize the split order, then take the specified number of keys from each split sampled, where each key is selected with the specified probability and possibly replaced by a subsequently selected key when the quota of keys from that split is satisfied.

指定者:

getSample 在接口中 InputSampler.Sampler<K,V>

抛出:

IOException