Reading nested JSON data encoded as a nested string with Aeson

Question

I have this weird JSON to parse containing nested JSON ... a string. So instead of

{\"title\": \"Lord of the rings\", \"author\": {\"666\": \"Tolkien\"}\"}"

I have

{\"title\": \"Lord of the rings\", \"author\": \"{\\\"666\\\": \\\"Tolkien\\\"}\"}"

Here's my (failed) attempt to parse the nested string using decode, inside an instance of FromJSON :

{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}

module Main where

import Data.Maybe
import GHC.Generics
import Data.Aeson
import qualified Data.Map as M

type Authors = M.Map Int String

data Book = Book
  {
    title :: String,
    author :: Authors
  }
  deriving (Show, Generic)

decodeAuthors x =  fromJust (decode x :: Maybe Authors)

instance FromJSON Book where
  parseJSON = withObject "Book" $ \v -> do
    t <- v .: "title"
    a <- decodeAuthors <?> v .: "author"
    return  $ Book t a

jsonTest = "{\"title\": \"Lord of the rings\", \"author\": \"{\\\"666\\\": \\\"Tolkien\\\"}\"}"

test = decode jsonTest :: Maybe Book

Is there a way to decode the whole JSON in a single pass ? Thanks !

Fyodor Soikin · Accepted Answer · 2021-04-30 20:52:26Z

3

A couple problems here.

First, your use of <?> is nonsensical. I'm going to assume it's a typo, and what you actually meant was <$>.

Second, the type of decodeAuthors is ByteString -> Authors, which means its parameter is of type ByteString, which means that the expression v .: "author" must be of type Parser ByteString, which means that there must be an instance FromJSON ByteString, but such instance doesn't exists (for reasons that escape me at the moment).

What you actually want is for v .: "author" to return a Parser String (or perhaps Parser Text), and then have decodeAuthors accept a String and convert it to ByteString (using pack) before passing to decode:

import Data.ByteString.Lazy.Char8 (pack)

decodeAuthors :: String -> Authors
decodeAuthors x = fromJust (decode (pack x) :: Maybe Authors)

(also note: it's a good idea to give you declarations type signatures that you think they should have. This lets the compiler point out errors earlier)

Edit:

As @DanielWagner correctly points out, pack may garble Unicode text. If you want to handle it correctly, use Data.ByteString.Lazy.UTF8.fromString from utf8-string to do the conversion:

import Data.ByteString.Lazy.UTF8 (fromString)

decodeAuthors :: String -> Authors
decodeAuthors x = fromJust (decode (fromString x) :: Maybe Authors)

But in that case you should also be careful about the type of jsonTest: the way your code is written, its type would be ByteString, but any non-ASCII characters that may be inside would be cut off because of the way IsString works. To preserve them, you need to use the same fromString on it:

jsonTest = fromString "{\"title\": \"Lord of the rings\", \"author\": \"{\\\"666\\\": \\\"Tolkien\\\"}\"}"

edited Apr 30, 2021 at 20:52

answered Apr 30, 2021 at 17:32

Fyodor Soikin

81.3k10 gold badges134 silver badges179 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Daniel Wagner Over a year ago

Don't use pack, use a UTF-8 encoder. Being careful about the distinction between bytes and codepoints is also the reason there's no FromJSON ByteString instance.

Fyodor Soikin Over a year ago

@DanielWagner But doesn't String swallow UTF-8 codepoints anyway?

Daniel Wagner Over a year ago

I don't know how to interpret that question. String is a list of Unicode codepoints. It certainly isn't limited to the first 256 codepoints, if that's the question. You ask about UTF-8 codepoints, but I don't know what that means, because "codepoint" is the term Unicode uses for talking about things independent of encoding. The compiler makes no promises about what encoding it uses for storing codepoints in memory (though in practice the only extant Haskell implementation--GHC--uses UCS-4).

Fyodor Soikin Over a year ago

Ok @DanielWagner, I added an amendment.

Collectives™ on Stack Overflow

Reading nested JSON data encoded as a nested string with Aeson

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related