jq to replace array key values from file

Question

I have a json file with this data:

{
  "data": [                                                                                                                                      
    {
      "name": "table",    
      "values": [    
        "This is old data",    
        "that needs to be",    
        "replaced."    
      ]    
    }    
  ]    
}

But my challege here is I need to replace that values array with words in a text or csv file:

this
this
this
is
is
an
an
array

My output needs to have (although I could probably get away with the words all on one line...):

      "values": [
        "this this this",
        "is is",
        "an an",
        "array"    
      ],

Is this possible with only jq? Or would I have to get awk to help out? I already started down the awk road with:

awk -F, 'BEGIN{ORS=" "; {print "["}} {print $2} END{{print "]"}}' filename

But I know there is still some work here...

And then I came across jq -Rn inputs. But I haven't figured out how or if I can get the desired result.

Thanks for any pointers.

What's the rule to separate strings here? A sequence of same word in CSV file is to be taken as one string? Is the rest of the JSON data unchanging? If yes you could directly produce JSON with awkwithout using jq. — pii_ke
– pii_ke, Commented Apr 20, 2021 at 14:03
Yes sequence of same word as one string... I believe in theory all the words could be in a single string, but that may a very long line. So the sequence of same words breaks it up. But then: How can I either pipe that into jq or read from a file to replace the values array? — Boyd
– Boyd, Commented Apr 20, 2021 at 14:13

Inian · Accepted Answer · 2021-04-20 14:04:09Z

3

Assuming you have a raw ASCII text file named file and an input JSON file, you could do

jq --rawfile txt file '.data[].values |= ( $txt | split("\n")[:-1] | group_by(.) | map(join(" ")) )' json

produces

{
  "data": [
    {
      "name": "table",
      "values": [
        "an an",
        "array",
        "is is",
        "this this this"
      ]
    }
  ]
}

edited Apr 20, 2021 at 14:04

answered Apr 20, 2021 at 13:55

Inian

87k15 gold badges166 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Thor Over a year ago

If you want to maintain the order of the original strings, you could replace file with a process substitution, e.g.: <(uniq -c file | while read n w; do yes "$w" | head -n$n | paste -sd' '; done)

dawg · Accepted Answer · 2021-04-21 12:00:41Z

You can use jq and awk.

Given:

$ cat file
{
    "data": [                                                         
        {
            "name": "table",
            "values": [ 
                "This is old data",
                "that needs to be",
                "replaced."
            ]    
        }    
    ]    
}


$ cat replacement
this
this
this
is
is
an
an
array

First create a string for the replacement array (awk is easy to use here):

ins=$(awk '!s {s=last=$1; next}
     $1==last{s=s " " $1; next} 
     {print s; s=last=$1}
     END{print s}' replacement | tr '\n' '\t')

Then use jq to insert into the JSON:

jq --rawfile txt <(echo "$ins") '.data[].values |= ( $txt | split("\t")[:-1] )' file 
{
  "data": [
    {
      "name": "table",
      "values": [
        "this this this",
        "is is",
        "an an",
        "array"
      ]
    }
  ]
}

You can also use ruby to process both files:

ruby -r json -e '
    BEGIN{ ar=File.readlines(ARGV[0])
                .map{|l| l.rstrip}
                .group_by{|e| e}
                .values
                .map{|v| v.join(" ")}
        j=JSON.parse(File.read(ARGV[1]))
    }   
    j["data"][0]["values"]=ar
    puts JSON.pretty_generate(j)' txt file

   # same output...

Collectives™ on Stack Overflow

jq to replace array key values from file

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related