JOLT transform flatten nested array with key value pairs

Question

I'm trying to transform the following JSON

{
  "data": {
    "keyvalues": [
      {
        "key": "location",
        "value": "sydney, au"
      },
      {
        "key": "weather",
        "value": "sunny"
      }
    ]
  },
  "food": {
    "name": "AllFoods",
    "date": "2018-03-08T09:35:17-03:00",
    "count": 2,
    "food": [
      {
        "name": "chocolate",
        "date": "2018-03-08T12:59:58-03:00",
        "rating": "10",
        "data": null
      },
      {
        "name": "hot dog",
        "date": "2018-03-08T09:35:17-03:00",
        "rating": "7",
        "data": {
          "keyvalues": [
            {
              "key": "topping",
              "value": "mustard"
            },
            {
              "key": "BUN type",
              "value": "toasted"
            },
            {
              "key": "servings",
              "value": "2"
            }
          ]
        }
      }
    ]
  }
}

Into, something simpler like this, using JOLT (in NIFI). Bringing the first top-level food attributes (name, date, count) into the header and then pulling the nested food array up, and then flattening out the food.data.keyvalues into a dict/hashmap.

{
  "header": {
    "location": "sydney, au",
    "weather": "sunny",
    "date": "2018-03-08",
    "count": 2
  },
  "foods": [
    {
      "name": "chocolate",
      "date": "2018-03-08T12:59:58-03:00",
      "rating": "10"
    },
    {
      "name": "hot dog",
      "date": "2018-03-08T09:35:17-03:00",
      "rating": "7",
      "topping": "mustard",
      "bun_type": "toasted",
      "servings": "2"
    }
  ]
}

I've got the first data part working, but I'm not sure how to handle the nested food element. The top level food info needs to move into the header section, and the second level food array, needs to flatten out the data.keyvalues.

Current spec... (only handles the top data.keyvalues)

[
  {
    "operation": "shift",
    "spec": {
      "data": {
        "keyvalues": {
          "*": { "@value": "@key" }
        }
      }
    }
  }
]

Milo S · Accepted Answer · 2018-11-04 19:28:11Z

3

Spec

[
  {
    "operation": "shift",
    "spec": {
      "data": {
        "keyvalues": {
          "*": {
            "value": "header.@(1,key)"
          }
        }
      },
      "food": {
        "date": "header.date",
        "count": "header.count",
        "food": {
          "*": {
            "name": "foods[&1].name",
            "date": "foods[&1].date",
            "rating": "foods[&1].rating",
            "data": {
              "keyvalues": {
                "*": {
                  "value": "foods[&4].@(1,key)"
                }
              }
            }
          }
        }
      }
    }
  }
]

answered Nov 4, 2018 at 19:28

Milo S

4,5861 gold badge22 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

lifebythedrop Over a year ago

Thanks Milo. I can't seem to get my head around this DSL yet... will need to study this some more.

Collectives™ on Stack Overflow

JOLT transform flatten nested array with key value pairs

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related