0

When i display my spark df, i run into a SparkRuntimeException. I am unsure of what it means or what I need to fix.

file_path = "/Volumes/filepath/file.xlsx"

df = spark.read \
    .format("com.crealytics.spark.excel") \
    .option("header", "true") \
    .option("inferSchema", "true") \
    .load(file_path)

display(df)

The following is the error:

SparkRuntimeException: [EXPRESSION_ENCODING_FAILED] Failed to encode a value of the expressions: if (assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, Header1), StringType, ObjectType(class java.lang.String)), true, false, true) AS Header1#41651 to a row. SQLSTATE: 42846
Improve this question
1
  • You have an Excel file that is not compatible with crealytics reader. Commented Jul 17 at 20:19
Related questions
0
java.lang.RuntimeException: scala.collection.immutable.$colon$colon is not a valid external type for schema of struct<513:int,549:int>
348
Difference between DataFrame, Dataset, and RDD in Spark
198
Convert spark DataFrame column to python list
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.