BASH: My variable's value is not displaying correctly inside a loop

Question

In the function below, I'm using JQ to parse the JSON data and get the httpOperations key/value pair.

#!/usr/bin/env bash
function batch_post() {
  SETTINGS_INPUT=$1 #! Variable that holds the argument value
  n=0
  :> "${SETTINGS_INPUT}"_req.tmp #! This will just create an empty file
  
  while read setting
  do
    # Get parameters that need to go in the batch put/post request
    url=$(echo $setting | ./jq -r '.url')
    body=$(echo $setting | ./jq -r '.response.body')

    # Get the request method
    method=$(cat settings.json | ./jq --arg URL ${url} '.settings[] | select(.pathPattern==$URL|tostring).httpOperations[] | select(.method == "PUT" or .method == "POST").method')
       
    # Save individual requests to be added to the batch request
    let n=n+1
  done <<< $(cat ${SETTINGS_INPUT} | ./jq -c '.')

  # Format individual requests in the expected batch json format
  cat "${SETTINGS_INPUT}"_req.tmp | ./jq -n '.requests |= [inputs]' > "${SETTINGS_LIST}"_batch_post_put_req
  # TODO: Validate this has the correct format and PUT/POST according to the settings
  cat "${SETTINGS_LIST}"_batch_post_put_req
  
  #! DO NOT execute API call with put/post request
}

and I'm passing a JSON file as my argument to this function, below is the content of the JSON file. Please note that this is not the full JSON data that I'm actually using.

   {
  "settings" : [ {
    "key" : "AccountingRules",
    "description" : "Accounting Rules settings",
    "context" : "Entity",
    "pathPattern" : "/accounting-rules",
    "httpOperations" : [ {
      "method" : "GET",
      "url" : "/settings/accounting-rules",
      "parameters" : [ ],
      "responseType" : {
        "$ref" : "#/definitions/AccountingRules",
        "definitions" : {
          "AccountingRules" : {
            "additionalProperties" : false,
            "type" : "object",
            "properties" : {
              "allowRevenueScheduleNegativeAmounts" : {
                "type" : "boolean"
              },
              "allowBlankAccountingCodes" : {
                "type" : "boolean"
              },
              "allowCreationInClosedPeriod" : {
                "type" : "boolean"
              },
              "allowUsageInClosedPeriod" : {
                "type" : "boolean"
              },
              "differentCurrencies" : {
                "type" : "boolean"
              }
            }
          }
        }
      }
    }, {
      "method" : "PUT",
      "url" : "/settings/accounting-rules",
      "parameters" : [ ],
      "requestType" : {
        "$ref" : "#/definitions/AccountingRules",
        "definitions" : {
          "AccountingRules" : {
            "additionalProperties" : false,
            "type" : "object",
            "properties" : {
              "allowRevenueScheduleNegativeAmounts" : {
                "type" : "boolean"
              },
              "allowBlankAccountingCodes" : {
                "type" : "boolean"
              },
              "allowCreationInClosedPeriod" : {
                "type" : "boolean"
              },
              "allowUsageInClosedPeriod" : {
                "type" : "boolean"
              },
              "differentCurrencies" : {
                "type" : "boolean"
              }
            }
          }
        }
      },
      "responseType" : {
        "$ref" : "#/definitions/AccountingRules",
        "definitions" : {
          "AccountingRules" : {
            "additionalProperties" : false,
            "type" : "object",
            "properties" : {
              "allowRevenueScheduleNegativeAmounts" : {
                "type" : "boolean"
              },
              "allowBlankAccountingCodes" : {
                "type" : "boolean"
              },
              "allowCreationInClosedPeriod" : {
                "type" : "boolean"
              },
              "allowUsageInClosedPeriod" : {
                "type" : "boolean"
              },
              "differentCurrencies" : {
                "type" : "boolean"
              }
            }
          }
        }
      }
    } ]
  }, {
    "key" : "AgingBuckets",
    "description" : "Aging Buckets",
    "context" : "Entity",
    "pathPattern" : "/aging-buckets",
    "httpOperations" : [ {
      "method" : "GET",
      "url" : "/settings/aging-buckets",
      "parameters" : [ ],
      "responseType" : {
        "$ref" : "#/definitions/AgingBucket",
        "definitions" : {
          "Bucket" : {
            "additionalProperties" : false,
            "type" : "object",
            "properties" : {
              "name" : {
                "type" : "string",
                "maxLength" : 100
              },
              "fromDaysPastDue" : {
                "type" : "integer"
              },
              "toDaysPastDue" : {
                "type" : "integer"
              }
            },
            "required" : [ "name" ]
          },
          "AgingBucket" : {
            "additionalProperties" : false,
            "type" : "object",
            "properties" : {
              "includeNegativeInvoice" : {
                "type" : "boolean"
              },
              "buckets" : {
                "type" : "array",
                "items" : {
                  "$ref" : "#/definitions/Bucket"
                }
              }
            }
          }
        }
      }
    } ]
}
]}

And whenever I'm displaying the output for my variable named "method", the output that's being returned is stacked on top of one another. To give you an example, the issue looks like this when displaying the $method value.

Display method below
"PUT"
"POST"
"PUT"
"POST"
"PUT"
"POST"
"PUT"
"POST"
"PUT"
"POST"
"PUT"
"POST"
"PUT"
"PUT"
"POST"
"PUT"
"POST"
"POST"
"PUT"
"POST"
Done displaying method

Wherein I'm expecting the output to be just a single "PUT" or "POST" only. Also, the thing I observed that's kind of weird for me is that this issue will show up when I try to display my variable executing "$ echo ${method}" command but when I remove the curly braces it sometimes works fine.

Please paste your script at shellcheck.net and try to implement the recommendations made there. — Cyrus
– Cyrus, Commented Oct 14, 2021 at 14:29
Quoting bash variables ("${varname}" instead of just ${varname}) helps dealing with values containing spaces, thus often solves issues related to them. — pmf
– pmf, Commented Oct 14, 2021 at 14:33

ilovetoask · Accepted Answer · 2021-10-18 05:58:38Z

1

I managed to make it work by replacing the single quote to double quotes in my JQ filter.

# Get the request method
method=$(cat settings.json | ./jq ".settings[] | select(.pathPattern==$URL|tostring).httpOperations[] | select(.method == \"PUT\" or .method == \"POST\").method")

I did not need to use the --arg option to define a JQ variable, what happened instead is I'm directly calling my bash variable and luckily it worked fine.

answered Oct 18, 2021 at 5:58

ilovetoask

315 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Weeble Over a year ago

Are you sure this is functioning as you intend? I'm struggling to see how it could work, for a number of reasons: 1. your variable was called url not URL, 2. your variable wasn't a quoted JSON string so this would likely cause a syntax error, 3. the problem described in my answer remains. Did this require more changes than are described here?

Weeble · Accepted Answer · 2021-10-15 16:38:43Z

0

select(.pathPattern==$URL|tostring)

This is unlikely to be what you intend. The pipe | operator has lower precedence than the equality == operator. This filter is equivalent to:

select((.pathPattern==$URL)|tostring)

...which will work out as either of these:

select(true|tostring)
select(false|tostring)

...which will in turn be:

select("true")
select("false")

...and perhaps you can see where this is going. All strings are considered truth-like values. So these filters will actually select all the settings every time!

answered Oct 15, 2021 at 16:38

Weeble

18.1k4 gold badges68 silver badges87 bronze badges

Collectives™ on Stack Overflow

BASH: My variable's value is not displaying correctly inside a loop

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related