Duplicate columns in spark read json

Question

I ran spark.read.json(json_file_path) on a json file with a flat structure.

I got the error message:

Reference 'col' is ambiguous, could be: col#29582L, col#29632

It is possible that this is due to another column with a name of different case : Col. How could this be resolved if I cannot modify the json data?

Community · Accepted Answer · 2017-05-23 12:32:08Z

15

After referring to the R case of the problem and searching for the configurations.

I found

spark.conf.set("spark.sql.caseSensitive", "true")

CommunityBot

11 silver badge

answered Feb 10, 2017 at 0:44

hangc

5,52112 gold badges39 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1