org.apache.spark.sql

Class SQLImplicits

java.lang.Object

org.apache.spark.sql.SQLImplicits

Direct Known Subclasses:

SQLContext.implicits$

public abstract class SQLImplicits
extends java.lang.Object

A collection of implicit methods for converting common Scala objects into DataFrames.

Since:

1.6.0

Constructor Summary

Constructors

Constructor and Description
SQLImplicits()

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract SQLContext	_sqlContext()
DataFrameHolder	intRddToDataFrameHolder(RDD<java.lang.Object> data) Creates a single column DataFrame from an RDD[Int].
<A extends scala.Product> DataFrameHolder	localSeqToDataFrameHolder(scala.collection.Seq<A> data, scala.reflect.api.TypeTags.TypeTag<A> evidence$7) Creates a DataFrame from a local Seq of Product.
<T> DatasetHolder<T>	localSeqToDatasetHolder(scala.collection.Seq<T> s, Encoder<T> evidence$5) Creates a Dataset from a local Seq.
DataFrameHolder	longRddToDataFrameHolder(RDD<java.lang.Object> data) Creates a single column DataFrame from an RDD[Long].
Encoder<boolean[]>	newBooleanArrayEncoder()
Encoder<java.lang.Object>	newBooleanEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newBooleanSeqEncoder()
Encoder<byte[]>	newByteArrayEncoder()
Encoder<java.lang.Object>	newByteEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newByteSeqEncoder()
Encoder<double[]>	newDoubleArrayEncoder()
Encoder<java.lang.Object>	newDoubleEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newDoubleSeqEncoder()
Encoder<float[]>	newFloatArrayEncoder()
Encoder<java.lang.Object>	newFloatEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newFloatSeqEncoder()
Encoder<int[]>	newIntArrayEncoder()
Encoder<java.lang.Object>	newIntEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newIntSeqEncoder()
Encoder<long[]>	newLongArrayEncoder()
Encoder<java.lang.Object>	newLongEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newLongSeqEncoder()
<A extends scala.Product> Encoder<java.lang.Object>	newProductArrayEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$3)
<T extends scala.Product> Encoder<T>	newProductEncoder(scala.reflect.api.TypeTags.TypeTag<T> evidence$1)
<A extends scala.Product> Encoder<scala.collection.Seq<A>>	newProductSeqEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$2)
Encoder<short[]>	newShortArrayEncoder()
Encoder<java.lang.Object>	newShortEncoder()
Encoder<scala.collection.Seq<java.lang.Object>>	newShortSeqEncoder()
Encoder<java.lang.String[]>	newStringArrayEncoder()
Encoder<java.lang.String>	newStringEncoder()
Encoder<scala.collection.Seq<java.lang.String>>	newStringSeqEncoder()
<A extends scala.Product> DataFrameHolder	rddToDataFrameHolder(RDD<A> rdd, scala.reflect.api.TypeTags.TypeTag<A> evidence$6) Creates a DataFrame from an RDD of Product (e.g.
<T> DatasetHolder<T>	rddToDatasetHolder(RDD<T> rdd, Encoder<T> evidence$4) Creates a Dataset from an RDD.
DataFrameHolder	stringRddToDataFrameHolder(RDD<java.lang.String> data) Creates a single column DataFrame from an RDD[String].
ColumnName	symbolToColumn(scala.Symbol s) An implicit conversion that turns a Scala Symbol into a Column.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SQLImplicits

public SQLImplicits()

Method Detail

_sqlContext

protected abstract SQLContext _sqlContext()

newProductEncoder

public <T extends scala.Product> Encoder<T> newProductEncoder(scala.reflect.api.TypeTags.TypeTag<T> evidence$1)

Since:

1.6.0

newIntEncoder

public Encoder<java.lang.Object> newIntEncoder()

Since:

1.6.0

newLongEncoder

public Encoder<java.lang.Object> newLongEncoder()

Since:

1.6.0

newDoubleEncoder

public Encoder<java.lang.Object> newDoubleEncoder()

Since:

1.6.0

newFloatEncoder

public Encoder<java.lang.Object> newFloatEncoder()

Since:

1.6.0

newByteEncoder

public Encoder<java.lang.Object> newByteEncoder()

Since:

1.6.0

newShortEncoder

public Encoder<java.lang.Object> newShortEncoder()

Since:

1.6.0

newBooleanEncoder

public Encoder<java.lang.Object> newBooleanEncoder()

Since:

1.6.0

newStringEncoder

public Encoder<java.lang.String> newStringEncoder()

Since:

1.6.0

newIntSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newIntSeqEncoder()

Since:

1.6.1

newLongSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newLongSeqEncoder()

Since:

1.6.1

newDoubleSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newDoubleSeqEncoder()

Since:

1.6.1

newFloatSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newFloatSeqEncoder()

Since:

1.6.1

newByteSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newByteSeqEncoder()

Since:

1.6.1

newShortSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newShortSeqEncoder()

Since:

1.6.1

newBooleanSeqEncoder

public Encoder<scala.collection.Seq<java.lang.Object>> newBooleanSeqEncoder()

Since:

1.6.1

newStringSeqEncoder

public Encoder<scala.collection.Seq<java.lang.String>> newStringSeqEncoder()

Since:

1.6.1

newProductSeqEncoder

public <A extends scala.Product> Encoder<scala.collection.Seq<A>> newProductSeqEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$2)

Since:

1.6.1

newIntArrayEncoder

public Encoder<int[]> newIntArrayEncoder()

Since:

1.6.1

newLongArrayEncoder

public Encoder<long[]> newLongArrayEncoder()

Since:

1.6.1

newDoubleArrayEncoder

public Encoder<double[]> newDoubleArrayEncoder()

Since:

1.6.1

newFloatArrayEncoder

public Encoder<float[]> newFloatArrayEncoder()

Since:

1.6.1

newByteArrayEncoder

public Encoder<byte[]> newByteArrayEncoder()

Since:

1.6.1

newShortArrayEncoder

public Encoder<short[]> newShortArrayEncoder()

Since:

1.6.1

newBooleanArrayEncoder

public Encoder<boolean[]> newBooleanArrayEncoder()

Since:

1.6.1

newStringArrayEncoder

public Encoder<java.lang.String[]> newStringArrayEncoder()

Since:

1.6.1

newProductArrayEncoder

public <A extends scala.Product> Encoder<java.lang.Object> newProductArrayEncoder(scala.reflect.api.TypeTags.TypeTag<A> evidence$3)

Since:

1.6.1

rddToDatasetHolder

public <T> DatasetHolder<T> rddToDatasetHolder(RDD<T> rdd,
                                      Encoder<T> evidence$4)

Creates a Dataset from an RDD.

Parameters:

rdd - (undocumented)

evidence$4 - (undocumented)

Returns:

(undocumented)

Since:

1.6.0

localSeqToDatasetHolder

public <T> DatasetHolder<T> localSeqToDatasetHolder(scala.collection.Seq<T> s,
                                           Encoder<T> evidence$5)

Creates a Dataset from a local Seq.

Parameters:

s - (undocumented)

evidence$5 - (undocumented)

Returns:

(undocumented)

Since:

1.6.0

symbolToColumn

public ColumnName symbolToColumn(scala.Symbol s)

An implicit conversion that turns a Scala Symbol into a Column.

Parameters:

s - (undocumented)

Returns:

(undocumented)

Since:

1.3.0

rddToDataFrameHolder

public <A extends scala.Product> DataFrameHolder rddToDataFrameHolder(RDD<A> rdd,
                                                             scala.reflect.api.TypeTags.TypeTag<A> evidence$6)

Creates a DataFrame from an RDD of Product (e.g. case classes, tuples).

Parameters:

rdd - (undocumented)

evidence$6 - (undocumented)

Returns:

(undocumented)

Since:

1.3.0

localSeqToDataFrameHolder

public <A extends scala.Product> DataFrameHolder localSeqToDataFrameHolder(scala.collection.Seq<A> data,
                                                                  scala.reflect.api.TypeTags.TypeTag<A> evidence$7)

Creates a DataFrame from a local Seq of Product.

Parameters:

data - (undocumented)

evidence$7 - (undocumented)

Returns:

(undocumented)

Since:

1.3.0

intRddToDataFrameHolder

public DataFrameHolder intRddToDataFrameHolder(RDD<java.lang.Object> data)

Creates a single column DataFrame from an RDD[Int].

Parameters:

data - (undocumented)

Returns:

(undocumented)

Since:

1.3.0

longRddToDataFrameHolder

public DataFrameHolder longRddToDataFrameHolder(RDD<java.lang.Object> data)

Creates a single column DataFrame from an RDD[Long].

Parameters:

data - (undocumented)

Returns:

(undocumented)

Since:

1.3.0

stringRddToDataFrameHolder

public DataFrameHolder stringRddToDataFrameHolder(RDD<java.lang.String> data)

Creates a single column DataFrame from an RDD[String].

Parameters:

data - (undocumented)

Returns:

(undocumented)

Since:

1.3.0