org.apache.spark.ml.image

Class ImageSchema

Object

org.apache.spark.ml.image.ImageSchema

public class ImageSchema
extends Object

:: Experimental :: Defines the image schema and methods to read and manipulate images.

Constructor Summary

Constructors

Constructor and Description
ImageSchema()

Method Summary

Modifier and Type	Method and Description
static StructType	columnSchema() Schema for the image column: Row(String, Int, Int, Int, Int, Array[Byte])
static byte[]	getData(Row row) Gets the image data
static int	getHeight(Row row) Gets the height of the image
static int	getMode(Row row) Gets the OpenCV representation as an int
static int	getNChannels(Row row) Gets the number of channels in the image
static String	getOrigin(Row row) Gets the origin of the image
static int	getWidth(Row row) Gets the width of the image
static String[]	imageFields()
static StructType	imageSchema() DataFrame with a single column of images named "image" (nullable)
static java.util.Map<String,Object>	javaOcvTypes() (Java-specific) OpenCV type mapping supported
static scala.collection.immutable.Map<String,Object>	ocvTypes() (Scala-specific) OpenCV type mapping supported
static Dataset<Row>	readImages(String path) Deprecated. use `spark.read.format("image").load(path)` and this `readImages` will be removed in 3.0.0. Since 2.4.0.
static Dataset<Row>	readImages(String path, SparkSession sparkSession, boolean recursive, int numPartitions, boolean dropImageFailures, double sampleRatio, long seed) Deprecated. use `spark.read.format("image").load(path)` and this `readImages` will be removed in 3.0.0. Since 2.4.0.
static String	undefinedImageType()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ImageSchema

public ImageSchema()

Method Detail

undefinedImageType

public static String undefinedImageType()

ocvTypes

public static scala.collection.immutable.Map<String,Object> ocvTypes()

(Scala-specific) OpenCV type mapping supported

Returns:

(undocumented)

javaOcvTypes

public static java.util.Map<String,Object> javaOcvTypes()

(Java-specific) OpenCV type mapping supported

Returns:

(undocumented)

columnSchema

public static StructType columnSchema()

Schema for the image column: Row(String, Int, Int, Int, Int, Array[Byte])

Returns:

(undocumented)

imageFields

public static String[] imageFields()

imageSchema

public static StructType imageSchema()

DataFrame with a single column of images named "image" (nullable)

Returns:

(undocumented)

getOrigin

public static String getOrigin(Row row)

Gets the origin of the image

Parameters:

row - (undocumented)

Returns:

The origin of the image

getHeight

public static int getHeight(Row row)

Gets the height of the image

Parameters:

row - (undocumented)

Returns:

The height of the image

getWidth

public static int getWidth(Row row)

Gets the width of the image

Parameters:

row - (undocumented)

Returns:

The width of the image

getNChannels

public static int getNChannels(Row row)

Gets the number of channels in the image

Parameters:

row - (undocumented)

Returns:

The number of channels in the image

getMode

public static int getMode(Row row)

Gets the OpenCV representation as an int

Parameters:

row - (undocumented)

Returns:

The OpenCV representation as an int

getData

public static byte[] getData(Row row)

Gets the image data

Parameters:

row - (undocumented)

Returns:

The image data

readImages

public static Dataset<Row> readImages(String path)

Deprecated. use `spark.read.format("image").load(path)` and this `readImages` will be removed in 3.0.0. Since 2.4.0.

Read the directory of images from the local or remote source

Parameters:

path - Path to the image directory

Returns:

DataFrame with a single column "image" of images; see ImageSchema for the details

Note:

If multiple jobs are run in parallel with different sampleRatio or recursive flag, there may be a race condition where one job overwrites the hadoop configs of another., If sample ratio is less than 1, sampling uses a PathFilter that is efficient but potentially non-deterministic.

readImages

public static Dataset<Row> readImages(String path,
                                      SparkSession sparkSession,
                                      boolean recursive,
                                      int numPartitions,
                                      boolean dropImageFailures,
                                      double sampleRatio,
                                      long seed)

Deprecated. use `spark.read.format("image").load(path)` and this `readImages` will be removed in 3.0.0. Since 2.4.0.

Read the directory of images from the local or remote source

Parameters:

path - Path to the image directory

sparkSession - Spark Session, if omitted gets or creates the session

recursive - Recursive path search flag

numPartitions - Number of the DataFrame partitions, if omitted uses defaultParallelism instead

dropImageFailures - Drop the files that are not valid images from the result

sampleRatio - Fraction of the files loaded

seed - (undocumented)

Returns:

DataFrame with a single column "image" of images; see ImageSchema for the details

Note:

If multiple jobs are run in parallel with different sampleRatio or recursive flag, there may be a race condition where one job overwrites the hadoop configs of another., If sample ratio is less than 1, sampling uses a PathFilter that is efficient but potentially non-deterministic.