I answered one of my previous questions, however, having fixed the issue, I am faced with an another issue regarding SQLServerException.
I am trying to read in data on an Azure SQLDB. I have successfully authenticated to the server however when I try to apply the function to read in data I get the following error:
com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near '5'
The following is more details on the error:
Py4JJavaError Traceback (most recent call last)
<command-3741352302548628> in readFromDb(processId, query)
3 try:
----> 4 jdbcDF = (spark.read
5 .format("jdbc")
/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
209 else:
--> 210 return self._df(self._jreader.load())
211
/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py in __call__(self, *args)
1303 answer = self.gateway_client.send_command(command)
-> 1304 return_value = get_return_value(
1305 answer, self.gateway_client, self.target_id, self.name
The code is as follows:
def readFromDb(processId, query):
try:
jdbcDF = (spark.read
.format("jdbc")
.option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
.option("url", f"jdbc:sqlserver://{DBServer}.database.windows.net;database={DBDatabase}")
.option("user", DBUser)
.option("query", query)
.option("password", DBPword)
.load()
)
return jdbcDF
except Exception as e:
writeToLogs(processId,LogType.Error, EventType.FailReadFromDb, LogMessage.FailReadFromDb, errorType = ErrorType.FailReadFromDb)
raise Error(f"{LogMessage.FailReadFromDb.value} ERROR: {e}")
except:
writeToLogs(processId,LogType.FailReadFromDb, EventType.FailReadFromDb, LogMessage.FailReadFromDb, errorType = ErrorType.FailReadFromDb)
raise Error(f"{LogMessage.FailReadFromDb.value}")
Can someone let me know what the code generally means and best approach to fix it?