How to run multiple line sql query using PySpark? [duplicate]

Question

spark.sql("Select acts.accountname, acts.county_state,loc.Town, acts.country from assure_crm_accounts acts inner join assure_crm_accountlocation loc on acts.GPAddressCode = loc.GPAddressCode").show(30, False)

I run into error when I use multiple line query using PySpark, is there a way to run multiple line queries using a single command?

Maybe it's handy to write it on 1 line, then after that use auto-reformatting shortcut in you editor. Then you see what's the best syntax! — JaFizz
– JaFizz, Commented Jan 20, 2020 at 16:27

blackbishop · Accepted Answer · 2020-01-20 18:17:15Z

5

You can also use triple quotes to write multiline string sql query as below:

spark.sql("""
Select acts.accountname, acts.county_state,loc.Town, acts.country 
from assure_crm_accounts acts 
inner join assure_crm_accountlocation loc 
on acts.GPAddressCode = loc.GPAddressCode
"""
)

answered Jan 20, 2020 at 18:17

blackbishop

32.8k11 gold badges61 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Joel Cochran Over a year ago

Thank you for this - it is the only reference I've found that doesn't require the \ character (which is a real pain for very large SQL statements).

feedy · Accepted Answer · 2020-01-20 16:25:34Z

3

Just put " \" at the end of each line:

spark.sql("Select acts.accountname, acts.county_state,loc.Town, acts.country from \
assure_crm_accounts acts inner join assure_crm_accountlocation loc on \
acts.GPAddressCode = loc.GPAddressCode").show(30, False)

answered Jan 20, 2020 at 16:25

feedy

1,1508 silver badges21 bronze badges

Collectives™ on Stack Overflow

How to run multiple line sql query using PySpark? [duplicate]

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related