2

How can i change the string type to datetime type on my elements of nested array (transaction_date)? Here are the spark dataframe that i have :

root
 |-- id
 |-- data: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- transaction: array (nullable = true)
 |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |-- timestamp: string (nullable = true)
 |    |    |    |    |-- transaction_date: string (nullable = true)

I had trying using this code, but it return error:

df = df.withColumn("transaction_date", df.data.transaction.transaction_date.cast(TimestampType()))
Improve this question
3
Related questions
2
Pyspark : Change nested column datatype
1
PySpark Conversion to Array Types
1
Convert multiple array of structs columns in pyspark sql
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.