bash idiom for url-decoding
Here is a bash idiom for url-decoding a string held in variabe x and assigning the result to variable y:
: "${x//+/ }"; printf -v y '%b' "${_//%/\\x}"
Unlike the accepted answer, it preserves trailing newlines during assignment. (Try assigning the result of url-decoding v%0A%0A%0A to a variable.)
It also is fast. It is 6700% faster at assigning the result of url-decoding to a variable than the accepted answer.
Caveat: It is not possible for a bash variable to contain a NUL. For example, any bash solution attempting to decode %00 and assign the result to a variable will not work.
Benchmark details
function.sh
#!/bin/bash
urldecode() { : "${*//+/ }"; echo -e "${_//%/\\x}"; }
x=%21%20
for (( i=0; i<5000; i++ )); do
y=$(urldecode "$x")
done
idiom.sh
#!/bin/bash
x=%21%20
for (( i=0; i<5000; i++ )); do
: "${x//+/ }"; printf -v y '%b' "${_//%/\\x}"
done
$ hyperfine --warmup 5 ./function.sh ./idiom.sh
Benchmark #1: ./function.sh
Time (mean ± σ): 2.844 s ± 0.036 s [User: 1.728 s, System: 1.494 s]
Range (min … max): 2.801 s … 2.907 s 10 runs
Benchmark #2: ./idiom.sh
Time (mean ± σ): 42.4 ms ± 1.0 ms [User: 40.7 ms, System: 1.1 ms]
Range (min … max): 40.5 ms … 44.8 ms 64 runs
Summary
'./idiom.sh' ran
67.06 ± 1.76 times faster than './function.sh'
If you really want a function ...
If you really want a function, say for readability reasons, I suggest the following:
# urldecode [-v var ] argument
#
# Urldecode the argument and print the result.
# It replaces '+' with SPACE and then percent decodes.
# The output is consistent with https://meyerweb.com/eric/tools/dencoder/
#
# Options:
# -v var assign the output to shell variable VAR rather than
# print it to standard output
#
urldecode() {
local assign_to_var=
local OPTIND opt
while getopts ':v:' opt; do
case $opt in
v)
local var=$OPTARG
assign_to_var=Y
;;
\?)
echo "$FUNCNAME: error: -$OPTARG: invalid option" >&2
return 1
;;
:)
echo "$FUNCNAME: error: -$OPTARG: this option requires an argument" >&2
return 1
;;
*)
echo "$FUNCNAME: error: an unexpected execution path has occurred." >&2
return 1
;;
esac
done
shift "$((OPTIND - 1))"
# Convert all '+' to ' '
: "${1//+/ }"
# We exploit that the $_ variable (last argument to the previous command
# after expansion) contains the result of the parameter expansion
if [[ $assign_to_var ]]; then
printf -v "$var" %b "${_//%/\\x}"
else
printf %b "${_//%/\\x}"
fi
}
Example 1: Printing the result to stdout
x='v%0A%0A%0A'
urldecode "$x" | od -An -tx1
Result:
76 0a 0a 0a
Example 2: Assigning the result of decoding to a shell variable:
x='v%0A%0A%0A'
urldecode -v y "$x"
echo -n "$y" | od -An -tx1
(same result)
This function, while not as fast as the idiom above, is still 1300% faster than the accepted answer at doing assignments due to no subshell being involved. In addition, as shown in the example's output, it preserves trailing newlines due to no command substitution being involved.
s/%/\x/g->s/%/\\x/g