Select values that match an element in bash array in JQ

Question

I need to run a script where I have an array of ids, and a json that contains elements with these ids and more data. I need to filter/select the values that matches the ones in the array. For some reason, when running directly on bash I got good results with some of the things I've tried, but not when running with a script. Not sure if this may be relevant or I did something wrong when passing from one to the other

For example, if I have a json file like

[
 {
   "id": 1,
   "value": 10
 },   
 {
   "id": 2,
   "value": 100
 },
 {
   "id": 3,
   "value": 5
 },
 {
   "id": 4,
   "value": 17
 },
 {
   "id": 5,
   "value": 84
 }
]

And the following array:

IDS=(1 2 3 4)

I need to retrieve the values 10, 100, 5 and 17.

I've tried some ways:

VALUES=$(jq --argjson IDS "$IDS" '.[] | select( $IDS =~ .id ) | .value' file.json)
VALUES=$(jq --argjson IDS "$IDS" '.[] | select( ${IDS[*]} =~ .id ) | .value' file.json)
# I have certainty that IDS will always have 4 elements, no more no less
VALUES=$(jq --argjson IDS "$IDS" '.[] | select( .id == ${IDS[0]} | .id == ${IDS[1]} | .id == ${IDS[2]} | .id == ${IDS[3]} ) | .value' file.json)

In all cases I'm getting `unexpected INVALID_CHARACTER (Unix shell quoting issues?)

Also, on the last case, when I replaced the ${IDS[n]} for a hardcoded number, it worked fine. Yet, I won't have the same numbers on each run, so I need that parametrized.

Any help would be great!

EDIT

Thanks everyone for the solutions. I kept the one I understood the most for now, but I'm really greatful with all

Note that $IDS in bash just becomes 1; it evaluates to ${IDS[0]}. I'd use --arg to pass in ${IDS[*]} and then split them into a JSON list. — Charles Duffy
– Charles Duffy, Commented Feb 27, 2023 at 14:41
And ${IDS[0]} isn't valid jq syntax. It's normal/expected you can't use bash syntax in a jq expression. — Charles Duffy
– Charles Duffy, Commented Feb 27, 2023 at 14:42
Also, all-caps variables are in a reserved namespace in shell used for system-meaningful variable names; you should use lowercase names for your own variables. — Charles Duffy
– Charles Duffy, Commented Feb 27, 2023 at 14:43
Insofar as this is a question about using shell arrays, answers can be found at stackoverflow.com/questions/26808855/… — peak
– peak, Commented Feb 27, 2023 at 15:16

0stone0 · Accepted Answer · 2023-02-27 15:03:59Z

2

Use --arg and then convert the Bash array to JSON integers:

($IDS | split(" ") | map(tonumber)) as $PIDS

Combine that with select() and index():

jq \
    --arg IDS "${IDS[*]}" \
    '($IDS | split(" ") | map(tonumber)) as $PIDS | 
        .[] | select([.id] | index($PIDS[])).value' \
input

Gives:

edited Feb 27, 2023 at 15:03

answered Feb 27, 2023 at 14:56

0stone0

45.5k6 gold badges53 silver badges82 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Charles Duffy Over a year ago

You need braces in ${IDS[*]}

Charles Duffy Over a year ago

This is doing an O(n) search of the id list for every input item, right? That's fine as long as we know it'll always be short, but wouldn't scale well if used in a place where the list could grow arbitrarily long.

Charles Duffy · Accepted Answer · 2023-02-27 17:47:42Z

2

One approach is to create a map of acceptable IDs and then do lookups within it:

ids=( 1 2 3 4 )

jq --arg idstr "${ids[*]}" '
  ([$idstr | split(" ")[] | {"key": ., "value": true}] | from_entries) as $idmap
  | .[] | select($idmap[.id | tostring]).value'

edited Feb 27, 2023 at 17:47

answered Feb 27, 2023 at 15:10

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Comments

Léa Gris · Accepted Answer · 2023-02-27 18:09:07Z

2

Alternatively, pass ids individual array elements as arguments to jq using --args "${ids[@]}" and read the values back into a bash array with mapfile:

#!/usr/bin/env bash

ids=( 1 2 3 4 )

mapfile -t values < <(

jq '
  (
    [$ARGS.positional[] | {"key": ., "value": true}] |
    from_entries
  ) as $idmap |
  .[] |
  select($idmap[.id | tostring]).value' \
  input.json \
  --args "${ids[@]}" 
)

# Debug show content of values array
declare -p values

Sample output:

declare -a values=([0]="10" [1]="100" [2]="5" [3]="17")

answered Feb 27, 2023 at 18:09

Léa Gris

20.2k4 gold badges39 silver badges52 bronze badges

Comments

oguz ismail · Accepted Answer · 2023-02-27 18:27:50Z

2

You're looking for something like this:

$ IDS=(1 2 3 4)
$ jq '.[] | select(IN(.id; $ARGS.positional[])) .value' file.json --jsonargs "${IDS[@]}"
10
100
5
17

edited Feb 27, 2023 at 18:27

answered Feb 27, 2023 at 14:56

oguz ismail

51.8k16 gold badges64 silver badges83 bronze badges

2 Comments

Léa Gris Over a year ago

Instead of passing a string then split it, one can pass individual bash array elements as json arguments: jq '.[] | select([.id] | index($ARGS.positional[])).value' --jsonargs "${IDS[@]}" which also saves the number conversion.

oguz ismail Over a year ago

@LéaGris I didn't know about --jsonargs, thank you

Collectives™ on Stack Overflow

Select values that match an element in bash array in JQ

4 Answers 4

2 Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related