I have a source data like this:
{A:123,B:"Hello world",C:[{D:123,E:"Spark"}]}
And i have a Object:
case class TestClass (A:Int;B:String;C:???)
val obj:Dataset[TestClass] = df.as[TestClass]
How should I define the type of C?
Add a comment
|
1 Answer
One option
case class Nested(D: Long, E: String)
case class TestClass (A: Long, B:String, C: Seq[Nested])
Usage:
spark.read.json(sc.parallelize(
Seq("""{"A": 123, "B": "Hello world", "C": [{"D": 123, "E": "Spark"}]}"""
))).as[TestClass].show
+---+-----------+-------------+
| A| B| C|
+---+-----------+-------------+
|123|Hello world|[[123,Spark]]|
+---+-----------+-------------+
