pyspark.sql.functions.to_timestamp#

pyspark.sql.functions.to_timestamp(col, format=None)[source]#

Converts a Column into pyspark.sql.types.TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to pyspark.sql.types.TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or column name: column values to convert.
format: literal string, optional: format to use to convert timestamp values.

Returns

Column: timestamp value as pyspark.sql.types.TimestampType type.

See also

pyspark.sql.functions.to_date()
pyspark.sql.functions.to_timestamp_ltz()
pyspark.sql.functions.to_timestamp_ntz()
pyspark.sql.functions.to_utc_timestamp()
pyspark.sql.functions.to_unix_timestamp()
pyspark.sql.functions.try_to_timestamp()
pyspark.sql.functions.date_format()

Examples

Example 1: Convert string to a timestamp

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.to_timestamp(df.t)).show()
+-------------------+
|    to_timestamp(t)|
+-------------------+
|1997-02-28 10:30:00|
+-------------------+

Example 2: Convert string to a timestamp with a format

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.to_timestamp(df.t, 'yyyy-MM-dd HH:mm:ss')).show()
+------------------------------------+
|to_timestamp(t, yyyy-MM-dd HH:mm:ss)|
+------------------------------------+
|                 1997-02-28 10:30:00|
+------------------------------------+