Copy PySpark Dataframe to MySQL

Question

I am trying to load a dataframe created with PySpark in DataBricks to MySql but it tells me:

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

df_videojuegos.select("Nombre", "Plataforma", "Año", "Genero", "Editorial", "Ventas NA", "Ventas EU", "Ventas JP", "Ventas Otros", "Ventas Global") \
    .write \
    .format("jdbc") \
    .option("url", "jdbc:mysql://127.0.0.1:3306/dezyre_db&useUnicode=true&characterEncoding=UTF-8&useSSL=false") \
    .option("driver", "com.mysql.jdbc.Driver").option("dbtable", "reportes") \
    .option("user", "root").option("password", "root") \
    .save()

I already have the connector installed in the cluster for MySql and these imported libraries:

from pyspark.sql import SparkSession
from pyspark.sql import Row

spark = SparkSession.builder \
    .config("spark.jars", "/home/hduser/mysql-connector-java-5.1.47/mysql-connector-java-5.1.47.jar") \
    .master("local") \
    .appName("PySpark_MySQL_test2") \
    .getOrCreate()

I don't know if the url that I have declared is really correct, or if I have the syntax wrong.

Install the connectors, create the connection with MySQL but when loading the data it gives me an error

Linexthe · Accepted Answer · 2022-11-07 09:53:45Z

1

Ok, the error was a newbie, I was trying to connect from DataBricks to a localhost! :C

answered Nov 7, 2022 at 9:53

Linexthe

113 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Community Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

Copy PySpark Dataframe to MySQL

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related