BASH manipulating text file into array

Question

I've got a text file (clients) that contains around 150 lines of information

Each line is similar to :

"2","USERID","ACCESSCODE" Eg:

"1","545ghu","7687686686868709ioo98968g"
"1","G2567u","54564df76786470976476987w"
"1","Y5po97","iuioubhjgjg768b79j9890980"

I want to grep this file, only find entries containing G2 or Y5 in the second column, remove all Double quotes and send the result to an array.

I can do this with

foo=( $(grep 'G2\|Y5' clients | sed 's/"//g') )

This result in the array foo which contains entries like :

foo[0] = 1,G2567u,54564df76786470976476987w

What I'd like is the results in foo to look like this:

G2567u (54564df76786470976476987w)

Can someone advise how to do this ?

Thanks

Your expected output looks strange due ( and ) and missing first number. What are you doing to do with this array? — anubhava
– anubhava, Commented Jan 29, 2018 at 15:33
arr=( $(...) ) is an antipattern -- in general, you shouldn't ever do it. F/e, if one of the words emitted by your command substitution is *, you'll get a list of filenames in your array. Use readarray, mapfile, read -a, or a while read loop with arr+=( "$item" ) instead. — Charles Duffy
– Charles Duffy, Commented Jan 29, 2018 at 15:33
I would rethink whether any of this should be done in bash, instead of a language with a proper CSV library and real data structures. — chepner
– chepner, Commented Jan 29, 2018 at 15:37
BTW, the data structure you're asking for is liable to be suboptimal. If your actual use case is, say, looking up the access code from the user ID, then you should have an associative array (what other languages might call a "dictionary" or "map") that maps the former to the latter. — Charles Duffy
– Charles Duffy, Commented Jan 29, 2018 at 15:40
My aim is to create a select menu of the result to allow us to select which USERID and Access code to use. Having the select options as ID (Code) is neater to view and the '1' etc isn't needed. — Tom
– Tom, Commented Jan 29, 2018 at 16:15

Charles Duffy · Accepted Answer · 2018-01-29 17:54:00Z

3

There's no need for any external command -- awk, sed or grep -- here; all the necessary primitives are available native to bash itself.

array=( )
while IFS='",' read -r num userid access_code _; do
  if [[ $userid =~ (G2|Y5) ]]; then
    array+=( "${userid} (${access_code})" )
  fi
done < clients

See:

BashFAQ #1 -- How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
BashPitfalls #50 - On why hosts=( $(aws ...) ) is an antipattern
The BashGuide on native regular expression matching

edited Jan 29, 2018 at 17:54

answered Jan 29, 2018 at 15:35

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Charles Duffy Over a year ago

It reads from stdin. You can read from a file named clients by putting < clients after the done, as you would redirect stdin for any other command.

karakfa · Accepted Answer · 2018-01-29 16:31:39Z

1

awk for text manipulation (and separation of concerns) with readarray to convert the output to bash array

to simplify parsing, set quote and comma as field delimiters, which will change the field index.

$ readarray ar < <(awk -F'[",]' '$5~/G2|Y5/{print $5,"("$8")"}' file)  

$ echo "${ar[0]}"
G2567u (54564df76786470976476987w)

also, perhaps you're looking for begins with instead of contains for the special values. In that case to eliminate false positives change the condition to $5~/^(G2|Y5)/

edited Jan 29, 2018 at 16:31

answered Jan 29, 2018 at 16:13

karakfa

67.8k8 gold badges45 silver badges59 bronze badges

1 Comment

karakfa Over a year ago

do you mean echo?

user7040344 · Accepted Answer · 2018-01-30 03:58:24Z

0

I try to use bash string slice, while loop to extract data into bash array.

declare -a arr

while IFS="\r\n" read -r line; do
    # replace "," to | as field delimiter
    line="${line//\",\"/|}"
    # remove "
    line="${line//\"/}"
    # remove first field via delimiter |
    line="${line#*|}"
    # extract userid
    userid="${line%%|*}"

    if [[ "${userid}" =~ (G2|Y5) ]]; then
        # extract access_code
        access_code="${line##*|}"
        # arr+=( "${userid} (${access_code})" )
        arr[${#arr[@]}]="${userid} (${access_code})"
    fi
done < <(echo "${clients}")
# done < /PATH/clients.txt

echo "${arr[@]}"
echo "${#arr[@]}"

answered Jan 30, 2018 at 3:58

user7040344

Collectives™ on Stack Overflow

BASH manipulating text file into array

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related