JSON decode nested field as Map[String, String] in Scala using circe

Question

A circe noob here. I am trying to decode a JSON string to case class in Scala using circe. I want one of the nested fields in the input JSON to be decoded as a Map[String, String] instead of creating a separate case class for it.

Sample code:

import io.circe.parser
import io.circe.generic.semiauto.deriveDecoder

case class Event(
  action: String,
  key: String,
  attributes: Map[String, String],
  session: String,
  ts: Long
)
case class Parsed(
  events: Seq[Event]
)

Decoder[Map[String, String]]

val jsonStr = """{
  "events": [{
      "ts": 1593474773,
      "key": "abc",
      "action": "hello",
      "session": "def",
      "attributes": {
          "north_lat": -32.34375,
          "south_lat": -33.75,
          "west_long": -73.125,
          "east_long": -70.3125
      }
  }]
}""".stripMargin

implicit val eventDecoder = deriveDecoder[Event]
implicit val payloadDecoder = deriveDecoder[Parsed]
val decodeResult = parser.decode[Parsed](jsonStr)
val res = decodeResult match {
  case Right(staff) => staff
  case Left(error) => error
}

I am ending up with a decoding error on attributes field as follows:

DecodingFailure(String, List(DownField(north_lat), DownField(attributes), DownArray, DownField(events)))

I found an interesting link here on how to decode JSON string to a map here: Convert Json to a Map[String, String]

But I'm having little luck as to how to go about it.

If someone can point me in the right direction or help me out on this that will be awesome.

While you certainly can, what do you prefer to have a Map[String, String] instead of a proper case class? Or are the attributes dynamic? — Luis Miguel Mejía Suárez
– Luis Miguel Mejía Suárez, Commented Jul 7, 2020 at 17:10
I am trying to convert this case class into an array of another case class which has key and value fields. I have tried converting the case class of attributes into a map in Scala before but it turns out to be ugly. So I was looking for a way where I can read this object as a Map[String,String] from JSON itself. — Arun Shyam
– Arun Shyam, Commented Jul 7, 2020 at 17:13
I do not understand what you meant with: "I am trying to convert this case class into an array of another case class which has key and value fields" - Why do you want to turn a case class into a Map? My original question remains, what does a Map gives you that a case class doesn't? — Luis Miguel Mejía Suárez
– Luis Miguel Mejía Suárez, Commented Jul 7, 2020 at 17:15
It's something that I need to do Luis. It feels easier for me to convert a Map[String, String] tp a list of case classes of type Attribute(key: String, value: String) rather than converting one case class into a list of some other case class given some of the fields may or may not be present in the attributes that are coming in this JSON. To answer your question, yes, these fields are dynamic and unfortunately I at this point can't ask our partner to change the traffic pattern. — Arun Shyam
– Arun Shyam, Commented Jul 7, 2020 at 17:19
Ok so with that context it is clear what you need to do. I would do this, first use List[Attribute] instead of Map[String, String] on your case class. Define your own explicit decoder for List[Attribute] and then use that to automatically derive the decoder of Event. I am on mobile right now, so I can't help with the code, but if when I got access to a computer you still haven't receive an answer I will give it a shoot. — Luis Miguel Mejía Suárez
– Luis Miguel Mejía Suárez, Commented Jul 7, 2020 at 17:24

C4stor · Accepted Answer · 2020-07-07 17:13:10Z

2

Let's parse the error :

DecodingFailure(String, List(DownField(geotile_north_lat), DownField(attributes), DownArray, DownField(events)))

It means we should look in "events" for an array named "attributes", and in this a field named "geotile_north_lat". This final error is that this field couldn't be read as a String. And indeed, in the payload you provide, this field is not a String, it's a Double.

So your problem has nothing to do with Map decoding. Just use a Map[String, Double] and it should work.

answered Jul 7, 2020 at 17:13

C4stor

8,0567 gold badges33 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Arun Shyam Over a year ago

Amazing! Thank you for the quick response. This worked for me. Apologies for the stupid mistake.

C4stor Over a year ago

It's ok :-) Circe errors get some getting used to, but they are in fact quite formidable tools even if they look a bit ugly !

Luis Miguel Mejía Suárez Over a year ago

@ArunShyam so you said the attributes are dynamic but are you sure they will always be numbers? I thought the idea of a Map[String, String] was because the values could also be anything that you would need to parse again latter according to their names.

Arun Shyam Over a year ago

Yes Luis they are dynamic but currently they are all coming in as double. You are right this may or may not be the case in the future which will definitely cause issues.

Luis Miguel Mejía Suárez · Accepted Answer · 2020-07-08 15:34:25Z

So you can do something like this:

final case class Attribute(
    key: String,
    value: String
)

object Attribute {
  implicit val attributesDecoder: Decoder[List[Attribute]] =
    Decoder.instance { cursor =>
      cursor
        .value
        .asObject
        .toRight(
          left = DecodingFailure(
            message = "The attributes field was not an object",
            ops = cursor.history
          )
        ).map { obj =>
          obj.toList.map {
            case (key, value) =>
              Attribute(key, value.toString)
          }
        }
    }
}

final case class Event(
    action: String,
    key: String,
    attributes: List[Attribute],
    session: String,
    ts: Long
)

object Event {
  implicit val eventDecoder: Decoder[Event] = deriveDecoder
}

Which you can use like this:

val result = for {
  json <- parser.parse(jsonStr).left.map(_.toString)
  obj <- json.asObject.toRight(left = "The input json was not an object")
  eventsRaw <- obj("events").toRight(left = "The input json did not have the events field")
  events <- eventsRaw.as[List[Event]].left.map(_.toString)
} yield events

// result: Either[String, List[Event]] = Right(
//   List(Event("hello", "abc", List(Attribute("north_lat", "-32.34375"), Attribute("south_lat", "-33.75"), Attribute("west_long", "-73.125"), Attribute("east_long", "-70.3125")), "def", 1593474773L))
// )

You can customize the Attribute class and its Decoder, so their values are Doubles or Jsons.

This is super Luis! Certainly generic and robust. I updated my code to work on these lines.

Collectives™ on Stack Overflow

JSON decode nested field as Map[String, String] in Scala using circe

2 Answers 2

4 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related