org.apache.spark.ml.clustering

Class PowerIterationClustering

Object

org.apache.spark.ml.clustering.PowerIterationClustering

All Implemented Interfaces:

DefaultParamsWritable, MLWritable

public class PowerIterationClustering
extends Object
implements DefaultParamsWritable

:: Experimental :: Power Iteration Clustering (PIC), a scalable graph clustering algorithm developed by Lin and Cohen. From the abstract: PIC finds a very low-dimensional embedding of a dataset using truncated power iteration on a normalized pair-wise similarity matrix of the data.

This class is not yet an Estimator/Transformer, use assignClusters method to run the PowerIterationClustering algorithm.

See Also:

Spectral clustering (Wikipedia), Serialized Form

Constructor Summary

Constructors

Constructor and Description
PowerIterationClustering()

Method Summary

Modifier and Type	Method and Description
Dataset<Row>	assignClusters(Dataset<?> dataset) Run the PIC algorithm and returns a cluster assignment for each input vertex.
static Params	clear(Param<?> param)
PowerIterationClustering	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
static Param<String>	dstCol()
Param<String>	dstCol() Name of the input column for destination vertex IDs.
static String	explainParam(Param<?> param)
static String	explainParams()
static ParamMap	extractParamMap()
static ParamMap	extractParamMap(ParamMap extra)
static <T> scala.Option<T>	get(Param<T> param)
static <T> scala.Option<T>	getDefault(Param<T> param)
static String	getDstCol()
String	getDstCol()
static String	getInitMode()
String	getInitMode()
static int	getK()
int	getK()
static int	getMaxIter()
static <T> T	getOrDefault(Param<T> param)
static Param<Object>	getParam(String paramName)
static String	getSrcCol()
String	getSrcCol()
static String	getWeightCol()
static <T> boolean	hasDefault(Param<T> param)
static boolean	hasParam(String paramName)
static Param<String>	initMode()
Param<String>	initMode() Param for the initialization algorithm.
static boolean	isDefined(Param<?> param)
static boolean	isSet(Param<?> param)
static IntParam	k()
IntParam	k() The number of clusters to create (k).
static PowerIterationClustering	load(String path)
static IntParam	maxIter()
static Param<?>[]	params()
static void	save(String path)
static <T> Params	set(Param<T> param, T value)
PowerIterationClustering	setDstCol(String value)
PowerIterationClustering	setInitMode(String value)
PowerIterationClustering	setK(int value)
PowerIterationClustering	setMaxIter(int value)
PowerIterationClustering	setSrcCol(String value)
PowerIterationClustering	setWeightCol(String value)
static Param<String>	srcCol()
Param<String>	srcCol() Param for the name of the input column for source vertex IDs.
static String	toString()
String	uid() An immutable unique ID for the object and its derivatives.
static Param<String>	weightCol()
static MLWriter	write()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter, maxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol

getWeightCol, weightCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Constructor Detail

PowerIterationClustering

public PowerIterationClustering()

Method Detail

load

public static PowerIterationClustering load(String path)

toString

public static String toString()

params

public static Param<?>[] params()

explainParam

public static String explainParam(Param<?> param)

explainParams

public static String explainParams()

isSet

public static final boolean isSet(Param<?> param)

isDefined

public static final boolean isDefined(Param<?> param)

hasParam

public static boolean hasParam(String paramName)

getParam

public static Param<Object> getParam(String paramName)

set

public static final <T> Params set(Param<T> param,
                                   T value)

get

public static final <T> scala.Option<T> get(Param<T> param)

clear

public static final Params clear(Param<?> param)

getOrDefault

public static final <T> T getOrDefault(Param<T> param)

getDefault

public static final <T> scala.Option<T> getDefault(Param<T> param)

hasDefault

public static final <T> boolean hasDefault(Param<T> param)

extractParamMap

public static final ParamMap extractParamMap(ParamMap extra)

extractParamMap

public static final ParamMap extractParamMap()

maxIter

public static final IntParam maxIter()

getMaxIter

public static final int getMaxIter()

weightCol

public static final Param<String> weightCol()

getWeightCol

public static final String getWeightCol()

k

public static final IntParam k()

getK

public static int getK()

initMode

public static final Param<String> initMode()

getInitMode

public static String getInitMode()

srcCol

public static Param<String> srcCol()

getSrcCol

public static String getSrcCol()

dstCol

public static Param<String> dstCol()

getDstCol

public static String getDstCol()

save

public static void save(String path)
                 throws java.io.IOException

Throws:

java.io.IOException

write

public static MLWriter write()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Returns:

(undocumented)

setK

public PowerIterationClustering setK(int value)

setInitMode

public PowerIterationClustering setInitMode(String value)

setMaxIter

public PowerIterationClustering setMaxIter(int value)

setSrcCol

public PowerIterationClustering setSrcCol(String value)

setDstCol

public PowerIterationClustering setDstCol(String value)

setWeightCol

public PowerIterationClustering setWeightCol(String value)

assignClusters

public Dataset<Row> assignClusters(Dataset<?> dataset)

Run the PIC algorithm and returns a cluster assignment for each input vertex.

Parameters:

dataset - A dataset with columns src, dst, weight representing the affinity matrix, which is the matrix A in the PIC paper. Suppose the src column value is i, the dst column value is j, the weight column value is similarity s,,ij,, which must be nonnegative. This is a symmetric matrix and hence s,,ij,, = s,,ji,,. For any (i, j) with nonzero similarity, there should be either (i, j, s,,ij,,) or (j, i, s,,ji,,) in the input. Rows with i = j are ignored, because we assume s,,ij,, = 0.0.

Returns:

A dataset that contains columns of vertex id and the corresponding cluster for the id. The schema of it will be: - id: Long - cluster: Int

copy

public PowerIterationClustering copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Parameters:

extra - (undocumented)

Returns:

(undocumented)

dstCol

public Param<String> dstCol()

Name of the input column for destination vertex IDs. Default: "dst"

Returns:

(undocumented)

getDstCol

public String getDstCol()

getInitMode

public String getInitMode()

getK

public int getK()

getSrcCol

public String getSrcCol()

initMode

public Param<String> initMode()

Param for the initialization algorithm. This can be either "random" to use a random vector as vertex properties, or "degree" to use a normalized sum of similarities with other vertices. Default: random.

Returns:

(undocumented)

k

public IntParam k()

The number of clusters to create (k). Must be > 1. Default: 2.

Returns:

(undocumented)

srcCol

public Param<String> srcCol()

Param for the name of the input column for source vertex IDs. Default: "src"

Returns:

(undocumented)