How to parse JSON data in Scala?

Question

I am new to Scala. I want to parse JSON data in scala.

I want to loop this data and in each iteration extract data for id,v,q, and t from values

I am using below code to parse it to JSON

import scala.util.parsing.json._

val data =
  """
{
  "timestamp":
  1518501114949
  , "values":
  [
  {
    "id":
    "abc"
    , "v":
    0
    , "q":
    true
    , "t":
    1518501114487
  }
  ,
  {
    "id":
    "xyz"
    , "v":
    15
    , "q":
    true
    , "t":
    1518501114494
  }
  ]
}
"""

val parsed = JSON.parseFull(data)

I am getting output as below

 Some(Map(timestamp -> 1.518501114949E12, values -> List(Map(id -> abc, v -> 0.0, q -> true, t -> 1.518501114487E12), Map(id -> xyz, v -> 15.0, q -> true, t -> 1.518501114494E12), Map(id -> klm, v -> 12.6999998, q -> true, t -> 1.518501114487E12), Map(id -> 901.Hotmelt.PSA.0759_PSAM01_Vac, v -> 1.0, q -> true, t -> 1.518501114494E12))))

but I don't know how to loop and fetch all values after that

and I am not understanding why timestamp is getting converted to E12 values

Possible duplicate of How to parse JSON in Scala using standard Scala classes? — SCouto
– SCouto, Commented Feb 13, 2018 at 11:48
The ujson library is the modern, best solution for parsing JSON. See my answer for more details. — Powers
– Powers, Commented Dec 17, 2020 at 16:19

SCouto · Accepted Answer · 2018-02-13 12:04:48Z

5

The problem is that the parseFull returns an Option with an Any inside, so you first need to get rid of that:

With this code below, you will keep the values:

val listAsAny = parsed match {
  case Some(e:Map[Any,Any]) => e("values")
  case None => println("Failed.")
}

But they still as Any, so you can transform it as follows:

val values = listAsAny.asInstanceOf[List[Map[String, Any]]]

Now values is a List of maps with the following values, and you can get the values inside as you will do with a regular List

List(Map(id -> abc, v -> 0.0, q -> true, t -> 1.518501114487E12), Map(id -> xyz, v -> 15.0, q -> true, t -> 1.518501114494E12))

For instance, to retrieve the ids you can do:

values.map(_("id"))

And the result will be:

List(abc, xyz)

answered Feb 13, 2018 at 12:04

SCouto

7,9465 gold badges37 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Tarun Khaneja Over a year ago

Is there any other method to parse data except json.parseFull()

Tarun Khaneja Over a year ago

and why timestamp is getting converted to E12

SCouto Over a year ago

Because it consider it to be a number. Check this answer, it may help stackoverflow.com/questions/4170949/…

SCouto Over a year ago

As a provisional solution, you can retrieve your timestamps as follows: values.map(_("t").toString.replaceAll("\\.", "").replaceAll("E12", ""))

Marcin Tarsier Over a year ago

@SCouto no, for christ sake don't use string replace to manipulate numbers! Just cast it to long with .toLong. Anyway, this accepted answer didn't solve this problem, that JSON.parseFull function for some cryptic reason turns all numbers into double, which is silly in case of large integers, where one can easily loose precision

Powers · Accepted Answer · 2020-12-17 16:16:15Z

1

The upickle library allows for a robust, elegant solution.

val parsed = ujson.read(data)
parsed("values").arr.map(_("id").str) // ArrayBuffer("abc", "xyz")

See here for a more detailed discussion on why upickle / ujson is the best Scala library for parsing JSON.

answered Dec 17, 2020 at 16:16

Powers

19.5k12 gold badges113 silver badges115 bronze badges

1 Comment

whatsinthename Over a year ago

Is there any way that it could be used for checking the schema and datatypes?

Marcin Tarsier · Accepted Answer · 2019-06-10 13:12:39Z

Regarding your second question

and I am not understanding why timestamp is getting converted to E12 values

this particular JSON parser treats all numbers as Double fractions, so that's why you get scientific notation with fraction × 10¹² (E12 suffix). There's an answer here on how to change this parser's default behaviour for numbers, namely one can implement own parser which would return Long instead of the default Double.

It makes more sense if you have large integer timestamps to parse, because you could easily start loosing precision (timestamps would be rounded if exceeded 2^51 ~= 4,5×10¹⁵, which is border of Double type fractional part precision). However, in your case numbers like 1518501114949 are 100 times smaller, so there's still some safe margin and probably casting the resulting Double to Long with .toLong method should be enough.

Collectives™ on Stack Overflow

How to parse JSON data in Scala?

3 Answers 3

5 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related