Pass date string as variable in spark sql

Question

I am unable to pass a date string in spark sql

When I run this

spark.sql(""" SELECT cast ('2021-04-12' as date) """)
> DataFrame[CAST(2021-04-12 AS DATE): date]

However I get error when I want to pass the date string as variable

 date_str = '2021-04-12'
 spark.sql(""" SELECT cast ({} as date) """.format(date_str))

 > AnalysisException: cannot resolve 'CAST(((2021 - 4) - 12) AS DATE)' 
 > due to data type mismatch: cannot cast int to date; line 1 pos 7;

I am not sure, how to pass that variable.

Anand Vidvat · Accepted Answer · 2021-04-12 07:13:02Z

2

you can just add single quotes to the query and it should work for you

spark.sql(""" SELECT cast ('{}' as date) """.format(date_str))

answered Apr 12, 2021 at 7:13

Anand Vidvat

1,0889 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sukumar · Accepted Answer · 2023-09-20 15:16:01Z

0

One more way to do that would be using fstring:

spark.sql(f""" SELECT cast ('{date_str}' as date) """)

answered Sep 20, 2023 at 15:16

Sukumar

614 bronze badges

Collectives™ on Stack Overflow

Pass date string as variable in spark sql

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related