How can I transform a list of Objects to a formatted String with object Fields using JQ

Question

I have an JSON array as below :

    [
      {
        "key" : "Name",
        "value" : "John"
      }, 
      { 
        "key" : "City",
        "value" : "New York"
       },
      {
        "key" : "Country",
        "value" : "USA"
      }
    ]

I need to convert it to a String like below :

--metadata "key": "Name", "value" : "John" --metadata "key": "City", "value" : "New York" --metadata "key": "Country", "value" : "USA"

This will be an input to a command in a python script.

I tried few things but didn't have luck :

I am trying to use a map function to do a transformation but I am not sure how to escape " for word key and value so this breaks and tried applying a join after this but getting and error.

. |map({"-- metadata key": .key, "value" : .value})

Output:

[
  {
    "-- metadata key": "Name",
    "value": "John"
  },
  {
    "-- metadata key": "City",
    "value": "New York"
  },
  {
    "-- metadata key": "Country",
    "value": "USA"
  }
]

If I append join(' ') to the above query, it throws below error:

jq: error: syntax error, unexpected INVALID_CHARACTER (Unix shell quoting issues?) at <top-level>,

I am very new to JQ so maybe not having enough knowledge to apply other functions, any help and explanation will be useful.

(Posting as a comment because I don't have time to explain how it works) Are you looking for something like this? — oguz ismail
– oguz ismail, Commented May 22, 2021 at 10:26
I'm curious -- are you creating a separate user account for every category of question you might encounter? ("techQueriesJQ JQ" is a rather unusual choice of name). Note that dividing your presence across multiple accounts means you build up reputation slower, and thus will have fewer privileges on the site than you would with a single shared account. — Charles Duffy
– Charles Duffy, Commented May 22, 2021 at 15:35
Need to remove "" at the start and end of the result string only and not removing other "" like for key, value as that is needed. — techQueriesJQ JQ
– techQueriesJQ JQ, Commented May 22, 2021 at 22:36

Joe Casadonte · Accepted Answer · 2021-05-22 15:51:05Z

This is a quick and dirty example:

$ jq -r '.[] | "--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\""' so8247.json | awk 'ORS=" "'
--metadata "key": "Name", "value": "John" --metadata "key": "City", "value": "New York" --metadata "key": "Country", "value": "USA"

(EDIT: see below for how to do this without awk)

The .[] bit just loops through the elements of the array; map(x) is equivalent to [.[] | x] so this is almost the same thing, just without throwing the results into an array that we don't want.

The rest of it just creates the string you're interested in. You can use \(x) to drop more JQ into the middle of the string. The output without '-r' and the awk would look like this:

$ jq '.[] | "--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\""' so8247.json
"--metadata \"key\": \"Name\", \"value\": \"John\""
"--metadata \"key\": \"City\", \"value\": \"New York\""
"--metadata \"key\": \"Country\", \"value\": \"USA\""

The '-r' gets rid of all of the extra quoting:

$ jq -r '.[] | "--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\""' so8247.json
--metadata "key": "Name", "value": "John"
--metadata "key": "City", "value": "New York"
--metadata "key": "Country", "value": "USA"

And the awk combines it into one line.

Using map gives you nearly the same thing, wrapped in an array:

$ jq -r 'map("--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\"")' so8247.json
[
  "--metadata \"key\": \"Name\", \"value\": \"John\"",
  "--metadata \"key\": \"City\", \"value\": \"New York\"",
  "--metadata \"key\": \"Country\", \"value\": \"USA\""
]

Which is pretty easy to strip off:

$ jq -r 'map("--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\"") | .[]' so8247.json
--metadata "key": "Name", "value": "John"
--metadata "key": "City", "value": "New York"
--metadata "key": "Country", "value": "USA"

Through that through awk and you're done:

$ jq -r 'map("--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\"") | .[]' so8247.json | awk 'ORS=" "'
--metadata "key": "Name", "value": "John" --metadata "key": "City", "value": "New York" --metadata "key": "Country", "value": "USA"

NOTE: Obviously I don't know how to do the awk bit in pure JQ; hopefully someone will show us how to do that, if it's possible.

EDIT: I just saw @oguz ismail's comment; using map and join gets rid of the need for awk:

$ jq -r 'map("--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\"") | join(" ")' so8247.json
--metadata "key": "Name", "value": "John" --metadata "key": "City", "value": "New York" --metadata "key": "Country", "value": "USA"

EDIT: Using Charles Duffy's suggestion of using from the comment of using '-j' to eliminate the map, join, and awk:

$ jq -j '.[] | "--metadata \"key\": \"\(.key)\", \"value\": \"\(.value)\""' so8247.json
--metadata "key": "Name", "value": "John"--metadata "key": "City", "value": "New York"--metadata "key": "Country", "value": "USA"

Consider switching from jq -r to jq -j, which will suppress the automatic newline after each record as a way to eliminate the need for awk while still streaming outputs one element at a time, thus being more efficient than the map+join approach.

Philippe · Accepted Answer · 2021-05-22 17:21:14Z

0

You can combine jq and bash scripts to get expected result :

#!/usr/bin/env bash

declare -a pyargs
while read -r line; do
    pyargs+=(--metadata "$line")
done < <(jq -r '.[]|"\"key\": \"\(.key)\", \"value\": \"\(.value)\""' json-input)

python "${pyargs[@]}"

answered May 22, 2021 at 17:21

Philippe

27.7k4 gold badges28 silver badges45 bronze badges

3 Comments

Philippe Over a year ago

@CharlesDuffy Thanks, how would you put in --metadata ?

techQueriesJQ JQ Over a year ago

In my case, I cannot put a bash script - I need to create a JQ command and feed into a different processor that will execute the jq.

Charles Duffy Over a year ago

Sorry, I missed that part. Yes, this loop is still the Right Thing with modern bash.

Collectives™ on Stack Overflow

How can I transform a list of Objects to a formatted String with object Fields using JQ

2 Answers 2

1 Comment

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related