How to parse a JSON object (which is a list) into simple Scala class objects in scala?

Question

I've spent too much time trying to make this work, I'm new to Scala.

Basically I make a request to an API and get the following response:

[
  {
    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
    "clientId": "account",
    "realm":"test-realm-uqrw"
    "name": "${client_account}",
    "rootUrl": "${authBaseUrl}",
    "baseUrl": "/realms/test-realm-uqrw/account/",
    "surrogateAuthRequired": false,
    "enabled": true,
    "alwaysDisplayInConsole": false,
    "clientAuthenticatorType": "client-secret",
    "defaultRoles": [
      "manage-account",
      "view-profile"
    ],
    "redirectUris": [
      "/realms/test-realm-uqrw/account/*"
    ],
    "webOrigins": [],
    "protocol": "openid-connect",
    "attributes": {},
    "authenticationFlowBindingOverrides": {},
    "fullScopeAllowed": false,
    "nodeReRegistrationTimeout": 0,
    "defaultClientScopes": [
      "web-origins",
      "role_list",

    ],

    "access": {
      "view": true,
      "configure": true,
      "manage": true
    }
  },
  {..another object of the same type, different values },
  {..another object of the same type, different values }
]

I just need to extract the "id" field from any of those objects(I match by the realm attribute later on). Is there a simple way to convert that json list into a List[] of Map[String, Any]? I say Any because the type of values are so varied - booleans, strings, maps, lists.

I've tried a couple of methods (internal tools) and Jackson(error: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of scala.collection.immutable.List(no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information) and the closest I got was to a weird list of Tuples giving me the incorrect result (because of incorrect processing).

What is a simple way to do this? Or am I destined to create a custom class for this API response? Or can I just walk down this JSON document (I just want one value from one of the objects in that array) and extract the value?

There is a lot of Scala libs above Jackson, to help Scala integration of json use cases: Circe, Json4s, Argonaut, LiftJson, Playjson — cchantep
– cchantep, Commented Mar 12, 2021 at 7:38

Ivan Kurchenko · Accepted Answer · 2021-03-12 13:47:01Z

One of the native and modern is Circe, in your case solution might look something like:

import io.circe._, io.circe.parser._, io.circe.generic.auto._, io.circe.syntax._

case class Response(
    id: String,
    clientId: String,
    realm: String,
    name: String,
    rootUrl: String,
    baseUrl: String,
    surrogateAuthRequired: Boolean,
    enabled: Boolean,
    alwaysDisplayInConsole: Boolean,
    clientAuthenticatorType: String,
    defaultRoles: List[String],
    redirectUris: List[String],
    webOrigins: List[String],
    protocol: String,
    fullScopeAllowed: Boolean,
    nodeReRegistrationTimeout: Int,
    defaultClientScopes: List[String],
    access: Access
)

case class Access(view: Boolean, configure: Boolean, manage: Boolean)

val json =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
       |    "clientId": "account",
       |    "realm":"test-realm-uqrw",
       |    "name": "client_account",
       |    "rootUrl": "authBaseUrl",
       |    "baseUrl": "/realms/test-realm-uqrw/account/",
       |    "surrogateAuthRequired": false,
       |    "enabled": true,
       |    "alwaysDisplayInConsole": false,
       |    "clientAuthenticatorType": "client-secret",
       |    "defaultRoles": [
       |      "manage-account",
       |      "view-profile"
       |    ],
       |    "redirectUris": [
       |      "/realms/test-realm-uqrw/account/*"
       |    ],
       |    "webOrigins": [],
       |    "protocol": "openid-connect",
       |    "fullScopeAllowed": false,
       |    "nodeReRegistrationTimeout": 0,
       |    "defaultClientScopes": [
       |      "web-origins",
       |      "role_list"
       |    ],
       |
       |    "access": {
       |      "view": true,
       |      "configure": true,
       |      "manage": true
       |    }
       |  }
       |]
       |""".stripMargin

println(parse(json).flatMap(_.as[List[Response]]))

Which will printout:

Right(List(Response(bde585ea-43ad-4e62-9f20-ea721193e0a5,account,test-realm-uqrw,client_account,authBaseUrl,/realms/test-realm-uqrw/account/,false,true,false,client-secret,List(manage-account, view-profile),List(/realms/test-realm-uqrw/account/*),List(),openid-connect,false,0,List(web-origins, role_list),Access(true,true,true))))

Scatie: https://scastie.scala-lang.org/5OpAUTjSTEWWTrH4X24vAg

The biggest advantage - unlike Jackson it's not based on runtime reflection, and instead, compile-time derivations.

UPDATE

As @LuisMiguelMejíaSuárez correctly suggested in comments section, if you would like to fetch only id field you can do it without full model parsing, like:

import io.circe._, io.circe.parser._

val json =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5"
       |  },
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a6"
       |  }
       |]
       |""".stripMargin

println(parse(json).map(_.hcursor.values.map(_.map(_.hcursor.downField("id").as[String]))))

Print out:

Right(Some(Vector(Right(bde585ea-43ad-4e62-9f20-ea721193e0a5), Right(bde585ea-43ad-4e62-9f20-ea721193e0a6))))

Scatie: https://scastie.scala-lang.org/bSSZdLPyTJWcup2KIb4zAw

But be careful - manual JSON manipulations, something usually used in edge cases. I'd suggest to go with model derivation even for simple cases.

@LuisMiguelMejíaSuárez Thank you so much, good point, I've missed that only one field required, I'll edit my answer.

Andriy Plokhotnyuk · Accepted Answer · 2021-03-14 09:39:41Z

Use jsoniter-scala FTW!

It is handy in derivation and the most efficient in runtime. Extraction of JSON values is where it shines at most.

Please add the following dependencies:

libraryDependencies ++= Seq(
  // Use the %%% operator instead of %% for Scala.js  
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core"   % "2.6.4",
  // Use the "provided" scope instead when the "compile-internal" scope is not supported  
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "2.6.4" % "compile-internal"
)

Then for just id values no need to define fields or data structures for other values.

Just define a simplest data structure and parse immediately to it:

import com.github.plokhotnyuk.jsoniter_scala.macros._
import com.github.plokhotnyuk.jsoniter_scala.core._
import java.util.UUID

val json = """[
             |  {
             |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
             |    "clientId": "account",
             |    "realm":"test-realm-uqrw",
             |    "name": "${client_account}",
             |    "rootUrl": "${authBaseUrl}",
             |    "baseUrl": "/realms/test-realm-uqrw/account/",
             |    "surrogateAuthRequired": false,
             |    "enabled": true,
             |    "alwaysDisplayInConsole": false,
             |    "clientAuthenticatorType": "client-secret",
             |    "defaultRoles": [
             |      "manage-account",
             |      "view-profile"
             |    ],
             |    "redirectUris": [
             |      "/realms/test-realm-uqrw/account/*"
             |    ],
             |    "webOrigins": [],
             |    "protocol": "openid-connect",
             |    "attributes": {},
             |    "authenticationFlowBindingOverrides": {},
             |    "fullScopeAllowed": false,
             |    "nodeReRegistrationTimeout": 0,
             |    "defaultClientScopes": [
             |      "web-origins",
             |      "role_list",
             |
             |    ],
             |
             |    "access": {
             |      "view": true,
             |      "configure": true,
             |      "manage": true
             |    }
             |  }
             |]""".stripMargin.getBytes("UTF-8")

case class Response(id: UUID)

implicit val codec: JsonValueCodec[List[Response]] = JsonCodecMaker.make

val responses = readFromArray(json)

println(responses)
println(responses.map(_.id))

Expected output:

List(Response(bde585ea-43ad-4e62-9f20-ea721193e0a5))
List(bde585ea-43ad-4e62-9f20-ea721193e0a5)

Feel free to ask for help here or in the gitter chat if it is need to handle your data differently or yet more efficiently.

John · Accepted Answer · 2021-03-14 17:31:15Z

1

You can use play json, it is very simple.

import play.api.libs.json._

case class Access(view: Boolean, configure: Boolean, manage: Boolean)
case class Response(
    id: String,
    clientId: String,
    realm: String,
    name: String,
    rootUrl: String,
    baseUrl: String,
    surrogateAuthRequired: Boolean,
    enabled: Boolean,
    alwaysDisplayInConsole: Boolean,
    clientAuthenticatorType: String,
    defaultRoles: List[String],
    redirectUris: List[String],
    webOrigins: List[String],
    protocol: String,
    fullScopeAllowed: Boolean,
    nodeReRegistrationTimeout: Int,
    defaultClientScopes: List[String],
    access: Access
)



val string =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
       |    "clientId": "account",
       |    "realm":"test-realm-uqrw",
       |    "name": "client_account",
       |    "rootUrl": "authBaseUrl",
       |    "baseUrl": "/realms/test-realm-uqrw/account/",
       |    "surrogateAuthRequired": false,
       |    "enabled": true,
       |    "alwaysDisplayInConsole": false,
       |    "clientAuthenticatorType": "client-secret",
       |    "defaultRoles": [
       |      "manage-account",
       |      "view-profile"
       |    ],
       |    "redirectUris": [
       |      "/realms/test-realm-uqrw/account/*"
       |    ],
       |    "webOrigins": [],
       |    "protocol": "openid-connect",
       |    "fullScopeAllowed": false,
       |    "nodeReRegistrationTimeout": 0,
       |    "defaultClientScopes": [
       |      "web-origins",
       |      "role_list"
       |    ],
       |
       |    "access": {
       |      "view": true,
       |      "configure": true,
       |      "manage": true
       |    }
       |  }
       |]
       |""".stripMargin

implicit val ac = Json.format[Access]
implicit val res = Json.format[Response]

println(Json.parse(string).asInstanceOf[JsArray].value.map(_.as[Response]))

to avoid exception-

val responseOpt = Json.parse(string) match {
        case JsArray(value: collection.IndexedSeq[JsValue]) => value.map(_.asOpt[Response])
        case _ => Seq.empty
      }

see : https://scastie.scala-lang.org/RBUHhxxIQAGcKgk9a9iwIA

Here is the doc : https://www.playframework.com/documentation/2.8.x/ScalaJson

edited Mar 14, 2021 at 17:31

answered Mar 12, 2021 at 7:49

John

2,7614 gold badges21 silver badges35 bronze badges

5 Comments

Saturnian Over a year ago

Can I just have the Response class hold just two fields - id and realm - and discard the other fields? Will the parse handle that?

John Over a year ago

@Saturnian yes, just remove other field from Response case class, See scastie.scala-lang.org/JNMH6IiAR0qq42FVbB4VGg . it will just create problem then when you declared any field in case class as non Option type, but that field is not present in json Object.

Saturnian Over a year ago

I'm curious though, why is res implicit?

John Over a year ago

@Saturnian this implicit will be used by macro to read and write json according to the Response case class. That is why it's not based on runtime reflection. At compile time it will create all the json mapping staffs (code) based on the implicit.

Tomer Shetah Over a year ago

This code is super unsafe. Both asInstanceOf and as might throw exception.

Tomer Shetah · Accepted Answer · 2021-03-14 07:34:39Z

1

Another option using play-json, is to define a path:

val jsPath = JsPath \\ "id"

Then to apply it:

jsPath(Json.parse(jsonString))

Code run at Scastie.

answered Mar 14, 2021 at 7:34

Tomer Shetah

8,5597 gold badges31 silver badges40 bronze badges

Collectives™ on Stack Overflow

How to parse a JSON object (which is a list) into simple Scala class objects in scala?

4 Answers 4

1 Comment

Comments

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related