Loop through JSON array shell script

Question

I am trying to write a shell script that loops through a JSON file and does some logic based on every object's properties. The script was initially written for Windows but it does not work properly on a MacOS.

The initial code is as follows

    documentsJson=""        
    jsonStrings=$(cat "$file" | jq -c '.[]')

    while IFS= read -r document; do

        # Get the properties from the docment (json string)
        currentKey=$(echo "$document" | jq -r '.Key')
        encrypted=$(echo "$document" | jq -r '.IsEncrypted')

        # If not encrypted then don't do anything with it
        if [[ $encrypted != true  ]]; then
            echoComment " Skipping '$currentKey' as it's not marked for encryption"
            documentsJson+="$document,"

            continue
        fi
//some more code

   done <<< $jsonStrings

When ran on a MacOs, the whole file is processed at once, so it does not loop through objects. The closest I got to making it work - after trying a lot of suggestions - is as follows:

jq -r '.[]' "$file" | while read i; do

    for config in $i ; do

    currentKey=$(echo "$config" | jq -r '.Key')
    echo "$currentKey"

    done

done

The console result is parse error: Invalid numeric literal at line 1, column 6

I just cannot find a proper way of grabbing the JSON object and reading its properties.

JSON file example

[
{
        "Key": "PdfMargins",
        "Value": {
            "Left":0,
            "Right":0,
            "Top":20,
            "Bottom":15
            }
        },
        {
        "Key": "configUrl",
        "Value": "someUrl",
        "IsEncrypted": true
    }
]

Thank you in advance!

Try putting the $jsonStrings in doublequotes: done <<< "$jsonStrings". — Alfe
– Alfe, Commented Nov 20, 2019 at 16:43
Damn, that fixed it. I cannot mark your comment as the accepted answer tho. — TheFunOne
– TheFunOne, Commented Nov 20, 2019 at 17:01
I gave it as an answer to help you get closure on this question ;-) — Alfe
– Alfe, Commented Nov 20, 2019 at 17:15
Strictly speaking, the quotes shouldn't be necessary. They work around bugs that exist prior to bash 4.4 (4.3 fixed some, but not all, of them). — chepner
– chepner, Commented Nov 20, 2019 at 17:33
I wonder how necessary the shell processing is; jq can do a lot more than you might think. For example, you can extract the encrypted elements of the list with just jq 'map(select(.IsEncrypted)) "$file"`. — chepner
– chepner, Commented Nov 20, 2019 at 17:37

Alfe · Accepted Answer · 2019-11-20 17:21:32Z

1

Try putting the $jsonStrings in doublequotes: done <<< "$jsonStrings"

Otherwise the standard shell splitting applies on the variable expansion and you probably want to retain the line structure of the output of jq.

You could also use this in bash:

while IFS= read -r document; do
   ...
done < <(jq -c '.[]' < "$file")

That would save some resources. I am not sure about making this work on MacOS, though, so test this first.

edited Nov 20, 2019 at 17:21

answered Nov 20, 2019 at 17:14

Alfe

60.2k21 gold badges117 silver badges172 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Loop through JSON array shell script

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related