Transform JSON array to object with jq

Question

I'm trying to transform array to object by specific key. It works fine without using stream, but not possible when stream is applied.

Data:

[
  {
    "id": "1",
    "userId": "fa51531d"
    }
    ,
    {
    "id": "2",
    "userId": "a167869a"
  }
]

I tried running this command but it throws an error.

jq -n --stream 'fromstream(1|truncate_stream(inputs)) | INDEX(.id)' test.json > result.json

Data above should be transformed to:

{
  "1": {
    "userId": "fa51531d",
    "id": "1"
  },
  "2": {
    "userId": "a167869a",
    "id": "2"
  },
}

I want to achieve the same result as with jq 'INDEX(.id) but I need to use stream (because of big JSON file).

Does the stream really contain the array start and end [ ... ] or is it a stream of objects { "id": "1", "userId": "fa51531d" } ... etc? — Ted Lyngmo
– Ted Lyngmo, Commented Jan 24, 2023 at 15:12

pmf · Accepted Answer · 2023-01-24 15:19:32Z

1

If you are trying to recreate the whole input object, the stream-based approach is rendered pointless. That said, using this approach, there's no need to truncate. So either replace 1 with 0:

jq -n --stream 'fromstream(0|truncate_stream(inputs)) | INDEX(.id)'

Or just omit it entirely (which reveals its futility):

jq -n --stream 'fromstream(inputs) | INDEX(.id)'

What would make more sense, is to output a stream of objects, each indexed as with INDEX. Maybe you were looking for this:

jq -n --stream 'fromstream(1|truncate_stream(inputs)) | {(.id):.}'

{
  "1": {
    "id": "1",
    "userId": "fa51531d"
  }
}
{
  "2": {
    "id": "2",
    "userId": "a167869a"
  }
}

answered Jan 24, 2023 at 15:19

pmf

38.3k3 gold badges31 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

wojteklu Over a year ago

Thanks @pmf for reply! Unfortunately, first two solutions are fine commands but never complete (probably too big file). Last one works immediately, but I want the result in one big object and each key is in separate object. Do you have more idea how to handle it?

pmf Over a year ago

@wojteklu Then try to reduce the stream: jq -n --stream 'reduce fromstream(1|truncate_stream(inputs)) as $o ({}; .[$o.id] = $o)'

Ted Lyngmo Over a year ago

@wojteklu Can you please verify that the stream really contains an array (with [ and ] around the individual items)? I asked the question in the comment section under your question but you never replied.

Ted Lyngmo · Accepted Answer · 2023-01-24 15:16:41Z

0

If your stream really looks like in your question, this should do:

jq 'INDEX(.id)' test.json

Output:

{
  "1": {
    "id": "1",
    "userId": "fa51531d"
  },
  "2": {
    "id": "2",
    "userId": "a167869a"
  }
}

answered Jan 24, 2023 at 15:16

Ted Lyngmo

124k7 gold badges93 silver badges155 bronze badges

Collectives™ on Stack Overflow

Transform JSON array to object with jq

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related