Scala Spark How can I convert a column array[string] to a string with JSON array in it?

Question

I have a column that contains an array of strings, I need to convert it into a JSON list of string? original value: [a,b,c] transformed value: "[\"a\",\"b\",\"c\"]"

Helder Pereira · Accepted Answer · 2021-01-30 23:53:51Z

6

You can combine struct, to_json and get_json_object:

import org.apache.spark.sql.functions._

val jsonArray = get_json_object(
  to_json(struct($"YOUR_COLUMN".as("data"))),
  "$.data"
)

df.select(jsonArray)

edited Jan 30, 2021 at 23:53

Helder Pereira

5,7713 gold badges39 silver badges55 bronze badges

answered Jul 26, 2018 at 12:04

user10138913

761 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

peeyush · Accepted Answer · 2019-09-29 11:50:44Z

2

Something like below could also help variableDF = variableDF.withColumn(f.name, to_json(struct(col(f.name))))

answered Sep 29, 2019 at 11:50

peeyush

2,9593 gold badges26 silver badges45 bronze badges

Collectives™ on Stack Overflow

Scala Spark How can I convert a column array[string] to a string with JSON array in it?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related