Create JSON schema from schema string Java-Spark

Question

I have the following schema:

root
  |-- id: string (nullable = true)
  |-- text: string (nullable = true)  
  |-- user: string (nullable = true)

How can I create StructType schema from this String?

I know that I can use .schema() method in my Dataset but I asking If its possible to create Schema from string.

Risking accusation of being accused of being snarky - read it line-by-line, extract parts you need, and create corresponding StructType / StructField structure. If you are the one who generates the string then it is XY-problem and much better ways of exporting and manipulating schema representation. In that case please edit your question, and the context. — Alper t. Turker
– Alper t. Turker, Commented Jul 24, 2018 at 11:20

lvnt · Accepted Answer · 2018-07-24 14:04:20Z

1

I was need something like that and i wrote a method, this able to creating the schema from string. But you must change this method for your scenario.

val schemaString = "val1#Int val2#String val3#Int"

val schema = StructType(schemaString.split(" ").map(fieldNameTypeStr => {
  val fieldNameType = fieldNameTypeStr.split("#")
  val dataType: DataType = fieldNameType(1) match {
    case "String" => StringType
    case "Int" => IntegerType
      /* other cases */
  }
  StructField(fieldNameType(0), dataType, true)
}))

answered Jul 24, 2018 at 14:04

lvnt

4973 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Create JSON schema from schema string Java-Spark

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related