Jolt spec to parse and transform JSON array

Question

This is my first time using Jolt and I am amazed at transformations it can perform. I followed the documentation and few posts online.

However, I am still have challenges with:

concatenating new values to existing values (url in this case)
moving concatenated value to right place
adding a default key-value pair if it doesn't exist in input JSON
Assigning JSON object to correct JSON array.

I have the following input

{
  "Body": [
    {
      "username": "some-user"
    },
    {
      "password": "*******"
    }
  ],
  "hostSource": "infos",
  "Host": [
    {
      "HOST_NAME": "xyz.com"
    },
    {
      "PORT": "9085"
    }
  ],
  "Headers": [
    {
      "Content-Type": "application/json"
    }
  ]
}

and my expected output is:

{
  "templateConfig": {
    "commonClientConfig ": {
      "item": [
        {
          "name": "Main API - BASICAUTH",
          "request": {
            "auth": {
              "type": "basic",
              "basic": [
                {
                  "key": "password",
                  "value": "*******",
                  "type": "string"
                },
                {
                  "key": "username",
                  "value": "some-user",
                  "type": "string"
                }
              ]
            },
            "method": "GET",
            "header": [
              {
                "key": "Content-Type",
                "value": "application/json",
                "type": "text"
              }
            ],
            "url": {
              "raw": "xyz.com/v9085/some-default-value"
            }
          }
        }
      ]
    }
  }
}

After referring few posts and documentation, I was able to get this far with the spec:

 [
  {
    "operation": "shift",
    "spec": {
      "Body": {
        "*": {
          "*": {
            "$": "commonClientConfig.item.request.auth.basic[&2].key",
            "@(1,&)": "commonClientConfig.item.request.auth.basic[&2].value"
          }
        }
      },
      "Headers": {
        "*": {
          "*": {
            "$": "commonClientConfig.item.request.header[&2].key",
            "@(1,&)": "commonClientConfig.item.request.header[&2].value"
          }
        }
      },
      "Host": {
        "*": {
          "*": {
            "@(1,&)": "commonClientConfig.item.request.url.raw"
          }
        }
      }
    }
    },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "content": "=join('/', @(2,commonClientConfig.item.request.url.raw))"
    }
  }    
]

I would really appreciate if someone can provide some guidance here with explanation.

Barbaros Özhan · Accepted Answer · 2021-09-09 06:33:20Z

1

You can use the following consecutive specs

[
  {
    "operation": "shift",
    "spec": {
      "#Main API - BASICAUTH": "name",
      "Body": {
        "#basic": "request.auth.type",
        "*": {
          "*": {
            "$": "request.auth.basic[&2].key",
            "@": "request.auth.basic[&2].value",
            "#string": "request.auth.basic[&2].type"
          }
        }
      },
      "Headers": {
        "#GET": "request.method",
        "*": {
          "*": {
            "$": "request.header[&2].key",
            "@": "request.header[&2].value",
            "#text": "request.header[&2].type"
          }
        }
      },
      "Host": {
        "*": {
          "*": {
            "@": "request.url"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "request": {
        "url": "=join('/', @(2,request.url))"
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "request": {
        "url": "=concat(@(1,url), /some-default-value)"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": "&",
      "request": {
        "*": "&1.&",
        "url": "&1.&.raw"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "@": "templateConfig.commonClientConfig.item[]"
    }
  }
]

where

join function is used to derive concatenated string from combining each elements of the array without repeatedly mentioning them, while concat is used just to handle the single concatenation of the default string.
Notice that the innermost common key is request for the arrays Body, Headers and Host those should be individually treated depending on the need for each, and they are finally accumulated under common nodes(templateConfig.commonClientConfig.item[]) within the last shift spec.

edited Sep 9, 2021 at 6:33

answered Sep 8, 2021 at 21:00

Barbaros Özhan

65.9k11 gold badges36 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Dwarrior Over a year ago

can u pls explain the url creation and movement within hierarchy, especially the last but one shift operation? Also, per original question, the url value needs a join with additional value that's not present in the input json, how to concatenate that?

Dwarrior Over a year ago

Hi @Barbaros - Thanks. Is there any tip/trick to use these tokens? for ex. If i have to comprehend, what this spec does -> "*": "&1.&", "url": "&1.&.raw"? The results are not obvious unless you are using the correct spec. On other hand, I was exploring mapneat library that uses Kotlin and it prints debug statements that tells what value is available for the variables.

Barbaros Özhan Over a year ago

@Dwarrior Jolt is just a library developed under github, and as far as I know there's no such IDE or framework for it, but you can use the site , which I use too, as a playground by trial and catch method to develop the comprehension. I like the way doing it :) Btw, check out my recent answers if you have time.

Dwarrior Over a year ago

sure thank u again for getting me so far, I will work on some more transformations to get a better feel of this library. Will open up new post in case I am unable to achieve something. I really like the power of this library but the examples are not that thorough in the documentation.

Dwarrior Over a year ago

I tried one more example and have posted question on that here - stackoverflow.com/questions/69211561. I would really appreciate if you can provide any feedback on the way I am approaching this problem using JOLT.

Collectives™ on Stack Overflow

Jolt spec to parse and transform JSON array

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related