How to put Array[Row] into JSON string

Question

I have Array[Row] called arr (I obtained it after df.collect()) that I want to pass in my JSON string as key and value pairs:

val result = """{"field1": "A", "arr": [""" + arr + """]}"""

It should be:

{"field1": "A", "arr": [
       {"name":"Ford", "model": "Fiesta"},
       {"name":"Ford", "model": "Mustang"},
       ...
]}

If I do it the way that I showed above, it will not work.

Should I iterate over this array and manually define each parameter?:

arr.get(i).get(arr.get(i).fieldIndex("field1")).toString()

Why don't you use to_json (with Dataframes) or toJSON with RDDs? What does the result of df.collect() look like ? — philantrovert
– philantrovert, Commented Feb 9, 2018 at 14:20
@philantrovert: If I do df.toJSON, then it returns Dataset[String]. What do I do with it? — Markus
– Markus, Commented Feb 9, 2018 at 15:02

Anahcolus · Accepted Answer · 2018-02-09 15:15:18Z

1

You should be doing as below by using .toJSON as suggested by philantrovert in comments of the question

val result = """{"field1":"A","arr":"""+df.toJSON.collectAsList()+"""}"""

If you are using arr variable then you can do

val arr = df.toJSON.collectAsList()
val result = """{"field1":"A","arr":"""+arr+"""}"""

answered Feb 9, 2018 at 15:15

Anahcolus

42.1k6 gold badges75 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1