How do I create a databricks table from a pandas dataframe?

Question

I have a pandas dataframe that I've created. This prints out fine, however I need to manipulate this in SQL.

I've run the following:

spark_df = spark.createDataFrame(df)

spark_df.write.mode("overwrite").saveAsTable("temp.testa")

pd_df = spark.sql('select * from temp.testa').toPandas()

But get an error:

AnalysisException: Database 'temp' not found;

Obviously I have not created a database, but not sure how to do this.

Can anyone advise how I might go about achieving what I need?

CHEEKATLAPRADEEP · Accepted Answer · 2020-09-11 05:38:08Z

2

The error message clearly says "AnalysisException: Database 'temp' not found;" database temp is not found. Once the database is created you can run the query without any issue.

To create a database, you can use the below command:

To create a database in SQL:

CREATE DATABASE <database-name>

Reference: Azure Databricks - SQL

answered Sep 11, 2020 at 5:38

CHEEKATLAPRADEEP

12.8k1 gold badge22 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I create a databricks table from a pandas dataframe?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related