JQ: Combine array with a list of objects from another file

Question

I need to append content in my primary file with information from a secondary file.

The primary file looks like this one:

[
  {
    "dynamic-parent-key-1": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1",
        "array-value-2"
      ]
    }
  },
  {
    "dynamic-parent-key-2": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1"
      ]
    }
  }
]

And the secondary file looks like this one:

{
  "fixedKey": [
    {
      "Key": "A",
      "Value": "abc"
    },
    {
      "Key": "B",
      "Value": "xyz"
    },
    {
      "Key": "C",
      "Value": "asd"
    }
  ]
}
{
  "fixedKey": [
    {
      "Key": "A",
      "Value": "aaa"
    },
    {
      "Key": "B",
      "Value": "bbb"
    },
    {
      "Key": "C",
      "Value": "ccc"
    }
  ]
}

The elements order is related. The first object "fixedKey" corresponds to "dynamic-parent-key-1" and so on. The first file is JSON compliant and the second is not, it is just a list of JSON objects saved on a common file.

I need an output file (or just update the first one) so it looks ideally like this:

{
  "master-key": {
    "dynamic-parent-key-1": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1",
        "array-value-2"
      ],
      "fixedKey": [
        {
          "Key": "A",
          "Value": "abc"
        },
        {
          "Key": "B",
          "Value": "xyz"
        },
        {
          "Key": "C",
          "Value": "asd"
        }
      ]
    },
    "dynamic-parent-key-2": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1"
      ],
      "fixedKey": [
        {
          "Key": "A",
          "Value": "aaa"
        },
        {
          "Key": "B",
          "Value": "bbb"
        },
        {
          "Key": "C",
          "Value": "ccc"
        }
      ]
    }
  }
}

But if that is not possible or too complicated, I could stick to first file structure and obtain something like this:

[
  {
    "dynamic-parent-key-1": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1",
        "array-value-2"
      ],
      "fixedKey": [
        {
          "Key": "A",
          "Value": "abc"
        },
        {
          "Key": "B",
          "Value": "xyz"
        },
        {
          "Key": "C",
          "Value": "asd"
        }
      ]
    }
  },
  {
    "dynamic-parent-key-2": {
      "key1": "value1",
      "key2": "value2",
      "array1": [
        "array-value-1"
      ],
      "fixedKey": [
        {
          "Key": "A",
          "Value": "aaa"
        },
        {
          "Key": "B",
          "Value": "bbb"
        },
        {
          "Key": "C",
          "Value": "ccc"
        }
      ]
    }
  }
]

How can I achieve this? I've been trying with map(), |=, + but I can't make it. Should I first prepare my secondary file to be JSON compliant or it is not necessary?

your secondary file is a stream of perfectly compliant JSONs, no need pre-processing it — Dmitry
– Dmitry, Commented Aug 3, 2020 at 22:08

luciole75w · Accepted Answer · 2020-08-04 00:28:33Z

1

The following script should work for you.

jq -n '{ "master-key": input | add | map_values(. + input) }' file1.json file2.json

The -n (--null-input) option disables the automatic read of input streams so that you can manually deal with each object (using functions like input or inputs). The first input call loads the array from your primary file and add converts this array to an object. Then map_values merges each sub-object with a new object loaded from the input stream (now on the secondary file) by subsequent input calls.

answered Aug 4, 2020 at 0:28

luciole75w

1,1176 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

M. Gleria Over a year ago

This works exactly as I expected. Thank you very much! Checking documentation to fully understand the script. Thanks again.

Collectives™ on Stack Overflow

JQ: Combine array with a list of objects from another file

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related