transform json to add array objects

Question

I need to transform an array by adding additional objects -

I have:

"user_id":"testuser"
"auth_token":"abcd"

I need:

"key":"user_id"
"value":"testuser"
"key":"auth_token"
"value":"abcd"

I have been using jq but cant figure out how to do it. Do i need to transform this into a multi-dimensional array first?

I have tried multiple jq queries but cant find the most suitable

When i try using jq i get

jq: error: syntax error, unexpected $end, expecting QQSTRING_TEXT or QQSTRING_INTERP_START or QQSTRING_END (Unix shell quoting issues?) at , line 1

The error is a parsing error (jq complains it reaches the end of the input while it was reading a string, which wasn't closed with the appropriate quote). I'm not sure whether it's your command or its input that it fails to parse, hard to tell when you didn't provide either. Probably the input though, I think command parsing errors don't follow that format. — Aaron
– Aaron, Commented Jun 21, 2019 at 13:08
jasonYardley - If your input is valid JSON, please show it; if the input is not intended to be valid JSON, please don't characterize it as such. Similarly please clarify your requirements regarding the output. — peak
– peak, Commented Jun 21, 2019 at 14:22
sorry this was my first time posting so my request came through unintentionally. basically i run a POST against a url that returns a credential. the credential is output as exaclty (minus changing the vlues): {"auth_token":"aaaa","id":"bbb"}. the problem im having is not inputting back into a different system as json, but rather transforming the json to include "key" and "value" as per my description. i have tried almost every jq argument i could find and spent many hours researching. i posted this to see if someone has done this before; transform the json to inclose extra objects inbetween — jasonYardley
– jasonYardley, Commented Jun 21, 2019 at 15:36

Jeff Mercado · Accepted Answer · 2019-06-21 17:32:38Z

1

Your input is not json, it's just a bunch of what could be thought of as key/value pairs. Assuming your json input actually looked like this:

{
    "user_id": "testuser",
    "auth_token": "abcd"
}

You could get an array of key/value pair objects using to_entries.

$ jq 'to_entries' input.json
[
    {
        "key": "user_id",
        "value": "testuser"
    },
    {
        "key": "auth_token",
        "value": "abcd"
    }
]

If on the other hand your input was actually that, you would need to convert it to a format that can be processed. Fortunately you could read it in as a raw string and probably parse using regular expressions or basic string manipulation.

$ jq -Rn '[inputs|capture("\"(?<key>[^\"]+)\":\"(?<value>[^\"]*)\"")]' input.txt

$ jq -Rn '[inputs|split(":")|map(fromjson)|{key:.[0],value:.[1]}]' input.txt

edited Jun 21, 2019 at 17:32

answered Jun 21, 2019 at 17:25

Jeff Mercado

136k34 gold badges267 silver badges282 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

jasonYardley Over a year ago

Thank you, that worked perfectly. my original output was in JSON i just pasted incorrectly. I tried so hard to find an expression that could do that and stumbled across a few. im still new to using jq but going to spend more time learning it now. thanks so much for your help!

Ilkiv Yuriy · Accepted Answer · 2019-06-21 12:48:09Z

1

You can use to_entries filter for that.

Here is jqplay example

answered Jun 21, 2019 at 12:48

Ilkiv Yuriy

1167 bronze badges

Comments

peak · Accepted Answer · 2019-06-22 06:11:32Z

1

Robust conversion of `key:value` lines to JSON.

If the key:value specifications would be valid JSON except for the missing punctuation (opening and closing braces etc), then a simple and quite robust approach to converting these key:value pairs to a single valid JSON object is illustrated by the following:

cat <<EOF | jq -nc -R '["{" + inputs + "}" | fromjson] | add' 
"user_id": "testuser"
"auth_token" : "abcd"
EOF

Output

{
  "user_id": "testuser",
  "auth_token": "abcd"
}

answered Jun 22, 2019 at 6:11

peak

119k21 gold badges185 silver badges218 bronze badges

Collectives™ on Stack Overflow

transform json to add array objects

3 Answers 3

1 Comment

Comments

Robust conversion of `key:value` lines to JSON.

Output

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Robust conversion of key:value lines to JSON.

Output

Comments

Your Answer

Sign up or log in

Post as a guest

Related

Robust conversion of `key:value` lines to JSON.