parse recursively nested Json structure with Play framework

Question

we are using Play framework 2.3.4. From one of the APIs we make a web service call to a third party service - structure of the returned response is dynamic and may change. Only sub-structure that's static within the response JSON is a particular element and nesting inside it. for e.g.

{
 "response": 
  {
   "someElement1": "",
   "element2": true,
   "buckets": [
     {
       "key": "keyvalue",
       "docCount": 10,
       "somethingElse": {
         "buckets": [
           {
             "key": "keyvalue1",
             "docCount": 5,
             "somethingElseChild": {
               "buckets": [
                 {
                   "key": "Tan",
                   "docCount": 1
                 }
               ]
             }
           },
           {
             "key": "keyvalue2",
             "docCount": 3,
             "somethingElseChild": {
               "buckets": [
                 {
                   "key": "Ban",
                   "docCount": 6
                 }
               ]
             }
           }
         ]
       }
     }
   ]
  }
}

we don't know how the response structure is going to look like but ONLY thing we know is that there will be "buckets" nested elements somewhere in the response and as you can see there are other nested "buckets" inside a top level "buckets" element. also please note that structure inside buckets array is also not clear and if there will be another sub bucket it's definite that sub bucket must be somewhere inside parent bucket - so that pattern is consistent.

what's the best way to parse such recursive structure and populate following Bucket class recursively?

case class Bucket(key:String,docCount, subBuckets: List[Bucket] )

First I was thinking to

val json = Json.parse(serviveResponse)
val buckets = (json \ "response" \\ "buckets")

but that will not bring bring buckets recursively and not right way to traverse.

Any ideas?

Would you mind depending on argonaut.io + github.com/alexarchambault/argonaut-shapeless ? — Reactormonk
– Reactormonk, Commented Feb 23, 2016 at 16:02
@Reactormonk : although I would have preferred to stay with play json given we already have play json heavily used elsewhere in the project. I am open to use argnaut for this particular requirement if argonaut is going to offer elegant/efficient solution to the problem at hand. I had only glanced over argonaut but never used it. would you mind sharing more detail on how can this be achieved using argonaut? — user2066049
– user2066049, Commented Feb 23, 2016 at 16:11
@Reactormonk : just added argonaut tag to the question - if you can share your answer. — user2066049
– user2066049, Commented Feb 23, 2016 at 16:18
Using argonaut is a side-tangent. You just want to recursively build your data structure, right? Write a recursive method. — Rex Kerr
– Rex Kerr, Commented Feb 23, 2016 at 16:37
@RexKerr : sure. my question is not around how to write a recursive method; but how to look for "buckets" recursively using JsPath (playframework.com/documentation/2.2.x/api/scala/… )- where I don't have knowledge of path upfront. — user2066049
– user2066049, Commented Feb 23, 2016 at 16:43

Kolmar · Accepted Answer · 2016-02-23 18:56:05Z

To make a Reads[T] for a recursive type T, you have to

define it as a lazy val,
use lazyRead where you need recursive parsing,
and manually pass to it the Reads[T] object or its derivative.

Of course you have to know what paths exactly the buckets element may appear at, and also account for it missing from any of those. You can use orElse to try several paths.

For your definition of Bucket, the Reads may look like this:

import play.api.libs.json._
import play.api.libs.functional.syntax._

implicit lazy val readBucket: Reads[Bucket] = (
  (__ \ "key").read[String] and
  (__ \ "docCount").read[Int] and
  (
    (__ \ "somethingElse" \ "buckets").lazyRead(Reads.list(readBucket)) orElse
    (__ \ "somethingElseChild" \ "buckets").lazyRead(Reads.list(readBucket)) orElse
    Reads.pure(List.empty[Bucket])
  )
) (Bucket.apply _)

You can simplify it a bit by extracting the common part to a function, e.g.:

def readsBucketsAt(path: JsPath): Reads[List[Bucket]] =
  (path \ "buckets").lazyRead(Reads.list(readBucket))

/* ... 
  readsBucketsAt(__ \ "somethingElse") orElse
  readsBucketsAt(__ \ "somethingElseChild") orElse
  Reads.pure(List.empty[Bucket])
... */

This example doesn't account for possible merging of several buckets arrays at different paths inside a single bucket. So if you need that functionality, I believe you'd have to define and use a play.api.libs.functional.Monoid instance for Reads[List[T]], or somehow combine the existing monoid instances for JsArray.

Rex Kerr · Accepted Answer · 2016-02-23 16:54:20Z

0

Parse recursively. Something like this (not tested):

case class Bucket(key: String, count: Int, sub: List[Bucket])

def FAIL = throw new Exception  // Put better error-handling here

def getBucket(js: JsValue): Bucket = js match {
  case o: JsObject =>
    val key = (o \ "key") match {
      case JsString(s) => s
      case _ => FAIL
    }
    val count = (o \ "docCount") match {
      case JsNumber(n) => n.toInt
      case _ => FAIL
    }
    val sub = (o \ "buckets") match {
      case a: JsArray => a.value.toList.map(getBucket)
      case _ => Nil
    }
    Bucket(key, count, sub)
  case _ => throw new Exception
}

answered Feb 23, 2016 at 16:54

Rex Kerr

168k27 gold badges325 silver badges411 bronze badges

Collectives™ on Stack Overflow

parse recursively nested Json structure with Play framework

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related