Processing an array of git commands inside bash script

Question

Good day to all:

I am writing a bash script to pull the data from git repos. I have an array to store the repository names:

declare -a arr=(
https://"$USER"@stash/repo1.git
https://"$USER"@stash/rep2.git
)

I process it as follows:

for i in "${arr[@]}"
do
    git clone "$i"
    ......
    git install
done

This works. Now I want to be able to optionally specify a git branch:

declare -a arr=(
    -b branch1 --single-branch https://"$USER"@stash/repo1.git
    https://"$USER"@stash/rep2.git
)

The array processing script fails with multiple ugly errors: - switch `b' requires a value - not valid identifier, etc

What is the right and simple approach to make it work?

Show us what you tried, show us what went wrong. Tell us what you want it to do. — ctrl-alt-delor
– ctrl-alt-delor, Commented Dec 26, 2018 at 17:36
In the second array, the first element is just -b, the second element is branch1, etc. — Charles Duffy
– Charles Duffy, Commented Dec 27, 2018 at 9:49

Paul Hodges · Accepted Answer · 2018-12-26 18:57:37Z

1

c.f. printf "%s\n" "${arr[@]}". Just because those args are on one line doesn't make them one element. :)

You are trying to do two things with one array.

Try this -

declare -a arr=(
   "https://$USER@stash/repo1.git#branch1"
   "https://$USER@stash/rep2.git"
)

for i in "${arr[@]}"
do  IFS='#' read url branch <<< "$i"
    if [[ -n "$branch" ]]
    then git clone -b "$branch" --single-branch "$url"
    else git clone "$url"
    :    ......
    git install
done

Warning - no chance to test this. Caveat Scriptor - and of course this means # can't be in your url anywhere.

answered Dec 26, 2018 at 18:57

Paul Hodges

16k1 gold badge24 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Den Potapov · Accepted Answer · 2018-12-28 16:29:42Z

0

Maybe it would work if you remove quotes around

git clone "$i"

So it would look like this:

git clone $i

I think that with quotes shell should treat all your line as one single argument instead of many:

git clone "-b branch1 --single-branch https://"$USER"@stash/repo1.git"

This looks wrong.

EDIT: @CharlesDuffy pointed to another problem I missed: you should also have quotes around the entire line in the array definition (but this won't work if you really need internal quotes around $USER, see example below):

declare -a arr=(
    "-b branch1 --single-branch https://$USER@stash/repo1.git"
    https://"$USER"@stash/rep2.git
)

I checked this on my local machine, and it seems to work.

$ ls
one/  test.sh*

$ ( cd one/ ; git branch )
* another_branch
  master

$ cat test.sh
#!/bin/bash

declare -a arr=(
"-b another_branch --single-branch one two"
)

for i in "${arr[@]}"
do
        git clone $i
done

$ ls
one/  test.sh*

$ ./test.sh
Cloning into 'two'...
done.

$ ls
one/  test.sh*  two/

$ ( cd two/ ; git branch )
* another_branch

$

EDIT2: This will only work if you can safely omit internal quotes around $USER. If you need them you should use eval inside the for loop and also quote the internal quotes in array declaration.

$ cat ./test.sh
#!/bin/bash

USER="username containing spaces"   # just for example!

git () {
        echo "$5"
}

declare -a arr=(
"-b branch1 --single-branch https://\"$USER\"@stash/repo1.git"
)

for i in "${arr[@]}"
do
        printf "without eval:\t"
        git clone $i
        printf "with eval:\t"
        eval "git clone $i"
done


$ ./test.sh
without eval:   https://"username
with eval:      https://username containing spaces@stash/repo1.git

This is yet another mistake that @CharlesDuffy found in my answer. Thanks Charles, I learn a lot from digging deeper into the problems you pointed!

edited Dec 28, 2018 at 16:29

answered Dec 27, 2018 at 9:23

Den Potapov

11 bronze badge

4 Comments

Charles Duffy Over a year ago

Even if you make it git clone $i, the first value in the second array is just -b, not -b branch1 --single-branch https://"$USER"@stash/repo1.git. So the proposed change is breaking with best practice (writing code that relies on string-splitting is responsible for more entries in BashPitfalls than any other single practice), and doesn't actually fix the OP's issue without additional changes not currently given in the answer.

Den Potapov Over a year ago

@CharlesDuffy True, I missed that author also don't have quotes around array elements. I will add it to my answer. Word splitting causes a lot of problems, because it is often used unintentionally and incorrectly. Here I used this particular feature intentionally and knowing what I am doing.

Charles Duffy Over a year ago

Quoted internal quotes aren't going to be treated as syntax unless you use eval -- as this code presently is implemented, they're going to be passed to git as literal data, as unquoted expansions only perform string-splitting and glob expansion, and don't perform quote removal. for i in "${arr[@]}"; do eval "git clone $i"; done could work, though. (The unquoted expansion would be safe with the specific data given, but it's not hard to come up with an example where using unquoted expansions to generate an argument list for eval breaks things).

Den Potapov Over a year ago

Thanks, Charles! I added it to the answer.

Collectives™ on Stack Overflow

Processing an array of git commands inside bash script

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related