2

I am writing a function that adds an element to the end of an array passed in parameter:

#@function add_elem_to_array: add an element to an array 
#in:
#1 name of the array
#2 element to add

add_elem_to_array()
{
   elem=$1  
   array=$2        
   index=${#array[@]} #get the index where to insert 

   eval "$array[$index]=$elem" #!!!! The problem is here
}

Could you please help me to figure out the solution?

12
  • Here's a page on dereferencing array elements in bash tldp.org/LDP/Bash-Beginners-Guide/html/sect_10_02.html Commented Jan 28, 2017 at 21:16
  • @GabrieleB-David thx, But in my case the name of the array is passed in param ; not the same case Commented Jan 28, 2017 at 21:18
  • Which version of bash? Do you have namevars? Commented Jan 28, 2017 at 21:18
  • 1
    BTW, using ${#array[@]} to get the next index to write to is buggy. Arrays can be sparse in bash - meaning you can have an array declare -a arr=( [15]=1 ), having only one item, with that item's index being 15, not 0. ${#arr[@]} won't give you the next index it's safe to insert at in such a case. Commented Jan 28, 2017 at 21:23
  • 1
    @GabrieleB-David, please avoid linking TLDP as a reference -- their documentation is undermaintained and (particularly in the case of the ABS) has a tendancy to showcase bad practices. BashFAQ #5 is a more actively-maintained reference covering use of arrays in detail, whereas BashFAQ #6 is a comprehensive discussion of associative arrays and both indirect assignment and indirect lookup (being closer to what the OP is actually looking for here). Commented Jan 28, 2017 at 21:33

4 Answers 4

4

I wouldn't use a function for this:

array+=("$elem")

appends an element.

If you really want to use a function and you have Bash 4.3 or newer, you can use a nameref:

add_elem_to_array () {
    local elem=$1
    local -n arr=$2
    arr+=("$elem")
}
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you it's quite as you said there is no need of function
3

Assuming bash 4.3 or newer, thus having namevars (declare -n / local -n):

add_elem_to_array() {
  local elem=$1 array_name=$2
  local -n array=$array_name
  array+=( "$elem" )
}

Supporting bash 3.x (particularly including 3.2, the oldest version in widespread use as of this writing):

add_elem_to_array() {
  local elem=$1 array_name=$2
  local cmd
  printf -v cmd '%q+=( %q )' "$array_name" "$elem"
  eval "$cmd"
}

That said -- given array+=( "$value" ) as an available syntax, there's little need for a function for the purpose, is there?

4 Comments

Is that eval statement safe (from a code injection standpoint), or does it require the name, the value or both to be known safe?
@Fred, this one's entirely safe due to the use of %q on all substitutions. Content will be escaped so that it evals back to its literal value, so if your name isn't a valid variable name you'll get a syntax error on that account, rather than arbitrary code execution.
@Fred I provided an answer that does not require eval. I tend avoid eval like the plague out of habit.
@CharlesDuffy OK this solves the problem thank you for the references and the valuable infos
2

Charles Duffy's answer works perfectly for Bash 4.3+, but there's no simple solution if you're using an older version of Bash (unless you wish to trifle with eval for some awful reason). However, it can indeed be done!

Here's what I whipped up:

## Arguments:
## $1 - the element to append
## $2 - the name of the array
append_to_array () {
    local -ia 'keys=(-1 "${!'$2'[@]}")';
    local IFS='';
    read -r -d '' -n ${#1} "$2"[${keys[${#keys[@]}-1]}+1] <<< "$1";
}

 


Explanation:

Indirection can be tricky and took me forever to learn, but it's powerful and fun so I figured I'd explain how everything fits together.

Let's use arr as the name of an array.
When you append elements to an array with something like arr+=(1) or arr+=("first element appended" "second element appended"), the indices(keys) of the elements in the array simply increment by 1 for each element. For example:

$ declare -a arr=(A)
$ arr+=(B)
$ arr+=(C D)
$ declare -p arr
declare -a arr='([0]="A" [1]="B" [2]="C" [3]="D")'
$ echo ${#arr[@]}
4

You can see the size of the array is equal to the array's next available index, but this is not always the case. Continuing on:

$ arr[7648]="E"
$ arr+=(F)
$ echo ${#arr[@]}
6
$ declare -p arr
declare -a arr='([0]="A" [1]="B" [2]="C" [3]="D" [7648]="E" [7649]="F")'

Line 1:
This is why in the first line of my function, I create an integer array, keys, from the indices of a ( ${!arr[@]} expands to the indices of arr. The last element in keys should be 1 less than the index we want to place the new element. However, if arr is unset or empty, ${!arr[@]} will expand to nothing, so I put -1 at the front of the keys to handle this.

Line 2:
Next up, we clear IFS (using local to avoid changing it outside of the function) to make sure any trailing or leading space characters in the appended element are preserved. Without clearing IFS, read and the here string operator <<< will strip leading and trailing space characters from "$1", which is undesirable.

Line 3:
In the third line, we use read to copy the value from "$1" into the array referenced by $2. The -r prevents read from processing/interpreting special characters in "$1" and the -d '' option sets the delimiter to the null character to allow our elements to contain newlines (I will come back to the -n ${#1} option.).
${#keys[@]}-1 evaluates to the index of the last element in keys, so ${keys[${#keys[@]}-1]}+1 grabs the last element of keys and adds one to it, forming our desired index to place "$1".

The read command can be used to write to elements in arrays, e.g. arr[2]="hi" could be replaced with read arr[2] <<< "hi", but read also works with indirect references to arrays, so we could also do nam=arr; read ${nam}[2] <<< "hi" or i=2; nam=arr; read ${nam}[$i] <<< "hi" and produce the same result. This is why read -r -d '' -n ${#1} ${2}[${keys[${#keys[@]}-1]}+1] <<< "$1" is able to append "$1" to the array referenced by $2.

Finally, -n ${#1} is required for reasons unknown to me. When I first wrote the script, every appended element had a newline character appended to it. I do not know why this is, so hopefully someone else can share some insight. So I just worked around this problem by limiting the number of characters read to the number of characters in "$1".

 

Improved version that can append any number of elements and sanity-checks arguments:

## WARNING: THE ARGUMENTS ARE NOT IN THE SAME ORDER AS THE ABOVE FUNCTION
## $1 - the name of the array
## $2 - the first element to append
## $3-... - optional; can append any number of elements to the array
array_append () {
    [[ $# -gt 1 && $1 =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]] || { 2>&1 echo "invalid args"; return 1; };
    local -ia 'k=(-1 "${!'$1'[@]}")';
    local n="$1" IFS='';
    local -i j=k[${#k[@]}-1] i=1 r=$#;
    while ((i<r)); do
        shift;
        read -r -d '' -n ${#1} "$n"[i+++j] <<< "$1";
    done;
}

7 Comments

To explain the need for the -n -- <<< implicitly appends a newline. < <(printf '%s\0' "$1") would avoid the issue, and cause read -d '' to correctly return success.
This is a good answer -- the only change I'd make is to quote more; by clearing IFS you're preventing string-splitting, but if a glob character made it into our arguments things could still get interesting. (It strikes me that quoting reflexively is something I do in the same manner in which you avoid eval -- because doing so as a consistently-applied habit avoids getting into sticky situations or showcasing practices that could be harmful if misapplied).
(not that read isn't returning success now, with -n, but when it was returning inaccurate results with an extra newline on the end, it was also exiting with a status of 1 since it wasn't finding the end-of-record delimiter it was being configured to look for).
@Charles Duffy Thanks for letting me know where that extra newline was coming from! Also, I have now updated the function to avoid any cases where a lack of quoting could cause an issue. Additionally, I added a function for appending and arbitrary number of elements.
At the risk of being pedantic -- ${#1} still needs to be quoted to handle the case where IFS contains digits.
|
0

Your problem is actually in the line where you determine the index.

   eval index=\${#${array}[@]} #get the index where to insert

You need to expand the variable containing the name of the array using the eval, then expand the expression to get its length. Escaping the first dollar sign inhibits the expansion of the array length request until after the eval.

The rest of the script appears to work as you intend. Here is a debugging version that I used so show what is happening:

add_elem_to_array()
{
   elem=$1
   array=$2
   eval index=\${#${array}[@]} #get the index where to insert

   echo "elem ='$elem'"
   echo "array='$array'"
   echo "index='$index'"

   eval "$array[$index]=$elem" #!!!! The problem is here
}

arr=(This is a test)
echo "arr = '${arr[@]}'"
add_elem_to_array "one" arr
echo "arr = '${arr[@]}'"
add_elem_to_array "two" arr
echo "arr = '${arr[@]}'"

1 Comment

This does correctly fix the index determination, but it's not fixing injection vulnerabilities.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.