Is it sensible to automatically URL encode query parameters in a string?

Question

Say I want to make the following request using curl:

https://api.foobar.com/widgets?begin=2018-09-10T01:00:00+01:00&object={"name":"barry"}

The URL encoded version of that string looks like this:

https://api.foobar.com/widgets?begin=2018-09-10T01%3A00%3A00%2B01%3A00&object=%7B%22name%22%3A%22barry%22%7D

Of course, when I'm making requests at the command line I would much rather look at the nicer looking (but not URL-valid) first version. I'm considering using a bash script to split out the different parts of the nice version, encode the relevant ones, and then glue it back together so I don't have to worry about it.

For example, after a couple of rounds of simple splitting on ?, &, and = I can easily get to:

https://api.foobar.com/widgets
begin
2018-09-10T01:00:00+01:00
object
{"name":"barry"}

And after that, URL encode the query string's two values and glue it all back together. I accept that any occurences of & and = in the query string will break this approach.

Is there anything else I should worry about that might make this a particularly stupid idea?

oguz ismail · Accepted Answer · 2018-09-20 18:31:09Z

5

Use --data-urlencode with --get

curl --data-urlencode 'begin=2018-09-10T01:00:00+01:00' --data-urlencode 'object={"name":"barry"}' --get 'http://api.foobar.com/widgets'

-G, --get When used, this option will make all data specified with -d, --data, --data-binary or --data-urlencode to be used in an HTTP GET request instead of the POST request that otherwise would be used. The data will be appended to the URL with a '?' separator.

edited Sep 20, 2018 at 18:31

answered Sep 20, 2018 at 18:22

oguz ismail

51.8k16 gold badges64 silver badges83 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Matt Over a year ago

Thanks @oguzismail, that inspired my script in the end (see my answer).

Matt · Accepted Answer · 2018-09-20 21:26:47Z

This is the script I came up with in the end, curl-encoded.sh:

#!/usr/bin/env bash
#
# Make HTTP request for given URL with query string URL encoded.
#

set -e

# Name args.
URL=$1

if [[ $URL = *"?"* ]]; then
  DOMAIN_PATH="${URL%%\?*}";
  QUERY_STRING="${URL#*\?}"
else
  DOMAIN_PATH=$URL
  QUERY_STRING=''
fi

# Split query string into key/value pairs.
IFS='&' read -ra PARAMETERS <<< "$QUERY_STRING"
for PARAMETER in "${PARAMETERS[@]}"; do
  URLENCODED_PARAMETERS=("${URLENCODED_PARAMETERS[@]}" "--data-urlencode" "$PARAMETER")
done

# Make request.
curl --silent "${URLENCODED_PARAMETERS[@]/#/}" "${@:2}" --get "$DOMAIN_PATH"

You call:

./curl-encoded.sh https://api.foobar.com/widgets?foo=bar&object={"name":"barry"}

And the URL that's fetched is:

https://api.foobar.com/widgets?foo=bar&object=%7B%22name%22%3A%22barry%22%7D

Collectives™ on Stack Overflow

Is it sensible to automatically URL encode query parameters in a string?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related