Build nested json data from variable with looping with jq using bash

Question

I'm trying to create a dynamic json data using jq, but I don't know how to write it when involving loop. I can do this in normal bash writing a sequence string

Here is the example code:

#!/bin/bash

# Here I declare 2 arrays just to demonstrate. Each values pairs between `disk_ids` and `volume_ids` cannot have the same value. If one has value, the other must null.

disk_ids=(111 null 222 333)
volume_ids=(null 444 null null)
json_query_data=""

# Now example I want to loop through the above 2 arrays and create json. In real application I already have the above arrays that is looping where I can access using $disk_id and $volume_id.

for disk_id in "${disk_ids[@]}"; do
  for volume_id in "${volume_ids[@]}"; do
    json_query_data=$(jq -n --argjson disk_id "$disk_id" --argjson volume_id "$volume_id" '{
                          devices: {
                                             sda: {"disk_id": $disk_id, "volume_id": $volume_id },
                                             sdb: {"disk_id": $disk_id, "volume_id": $volume_id },
                                             sdc: {"disk_id": $disk_id, "volume_id": $volume_id },
                                             sdd: {"disk_id": $disk_id, "volume_id": $volume_id },
                                          }}')

  done
done

As you can see that is definitely NOT the output that I want, and my code writing logic is not dynamic. The final output should produce the following json when I echo "${json_query_data}":

{
    devices: {
               sda: {"disk_id": 111, "volume_id": null },
               sdb: {"disk_id": null, "volume_id": 444 },
               sdc: {"disk_id": 222, "volume_id": null },
               sdd: {"disk_id": 333, "volume_id": null },
}}

I have not seen any example online regarding looping with variable when creating json data with jq. Appreciate if someone can help. Thanks.

UPDATE:

I must use for loop inside bash to create the json data. Because the sample array disk_ids and volume_ids that I provided in the code were just example, In real application I already able to access the variable $disk_id and $volume_id for each for loop counter. But how do I use this variables and create the json output that fill up all the data above inside that for loop?

The json example is taken from: linode API here

pmf · Accepted Answer · 2022-12-17 12:50:11Z

4

The looping/mapping can also be accomplished in jq:

#!/bin/bash

disk_ids=(111 null 222 333)
volume_ids=(null 444 null null)

jq -n --arg disk_ids "${disk_ids[*]}" --arg volume_ids "${volume_ids[*]}" '
  [$disk_ids, $volume_ids | . / " " | map(fromjson)]
  | transpose | {devices: with_entries(
      .key |= "sd\([. + 97] | implode)"
      | .value |= {disk_id: first, volume_id: last}
    )}
'

Demo

Or, if you can already provide the letters in the same way:

#!/bin/bash

disk_ids=(111 null 222 333)
volume_ids=(null 444 null null)
letters=(a b c d)

jq -n --arg disk_ids "${disk_ids[*]}" --arg volume_ids "${volume_ids[*]}" --arg letters "${letters[*]}" '
  [$disk_ids, $letters, $volume_ids | . / " " ] | .[0,2] |= map(fromjson)
  | transpose | {devices: with_entries(
      .key = "sd\(.value[1])"
      | .value |= {disk_id: first, volume_id: last}
    )}
'

Demo

Output:

{
  "devices": {
    "sda": {
      "disk_id": 111,
      "volume_id": null
    },
    "sdb": {
      "disk_id": null,
      "volume_id": 444
    },
    "sdc": {
      "disk_id": 222,
      "volume_id": null
    },
    "sdd": {
      "disk_id": 333,
      "volume_id": null
    }
  }
}

UPDATE:

I must use for loop inside bash to create the json data.

If you insist on doing this in a bash loop, how about:

#!/bin/bash

disk_ids=(111 null 222 333)
volume_ids=(null 444 null null)
json='{"devices": {}}'

for i in ${!disk_ids[@]}
do
  json="$(
    jq --argjson disk_id "${disk_ids[$i]}" --argjson volume_id "${volume_ids[$i]}" '
      .devices |= . + {"sd\([length + 97] | implode)": {$disk_id, $volume_id}}
    ' <<< "$json"
  )"
done
echo "$json"

Or, with letters included:

#!/bin/bash

disk_ids=(111 null 222 333)
volume_ids=(null 444 null null)
letters=(a b c d)
json='{"devices": {}}'

for i in ${!disk_ids[@]}
do
  json="$(
    jq --argjson disk_id "${disk_ids[$i]}" --argjson volume_id "${volume_ids[$i]}" --arg letter "${letters[$i]}" '
      .devices["sd\($letter)"] += {$disk_id, $volume_id}
    ' <<< "$json"
  )"
done
echo "$json"

edited Dec 17, 2022 at 12:50

answered Dec 17, 2022 at 6:05

pmf

38.3k3 gold badges31 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

Kalib Zen Over a year ago

What if I want to use for loop? Because the sample data that I wrote there were just example, Can this be done in a for loop where each loop I have the value of disk_id and volume_id respectively? If I use this solution, I will not be able to use the data $disk_id and $volume_id in my original code.

pmf Over a year ago

@KalibZen They use curly braces with objects. Note the object fields, e.g. "disk_id"

Kalib Zen Over a year ago

Thank you I think I wrote a wrong example. I will redo

Kalib Zen Over a year ago

Updated my code thank you for correcting that one. I wish to see example how to create the json query inside for loop. I must use for loop as the variable $disk_id and $volume_id are already generated in for loop for me to use. Can we build that json file straight from the data using jq?

pmf Over a year ago

@KalibZen You can even shortcut |= . + to +=. See my update.

|

Collectives™ on Stack Overflow

Build nested json data from variable with looping with jq using bash

1 Answer 1

14 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

14 Comments

Your Answer

Sign up or log in

Post as a guest

Related