Read CSV file in a Shell Script till End Of Line Without Header

Question

I have a file called as 'readfile.csv' with following contents.

Name    Id        VAl       Number    IP
James,007,$500,111-111-111,111-000-000
Bond,700,$900,100-000-999, 666-999-000

Here Name, Id, Val, Number and IP are heading in the csv file and the second line i.e., FirstVal, SecondVal etc are the values of those headers. I want to read only the values and not headers and display it in console. How do I do it? And also I want to read values till end of line/end of file as I don't know how many columns can be present in the file. It can be 5 or 6 columns.

Right now I am able to read a single line without header as below in shell script. But this does not help my condition mentioned here.

#!/bin/bash

val1=( $(cut -d ',' -f1 readfile.csv ) )
printf "%s\n" "${val1[0]}"
val2=( $(cut -d ',' -f2 readfile.csv ) )
printf "%s\n" "${val2[0]}"
val3=( $(cut -d ',' -f3 readfile.csv ) )
printf "%s\n" "${val3[0]}"

I actually want to append these values to other script. For example, I have a script called as runfile.sh which appends the values from the awk and opens a file location through http.

Example, 'runfile.sh' is storing this value:

'https://my_win_loc/personinfo'

Now runfile will read the csv file and append each value to this url, like this:

'https://my_win_loc/personinfo/James'
'https://my_win_loc/personinfo/Bond'

Please help me in resolving this. Thanks in advance.

William Pursell · Accepted Answer · 2019-03-12 15:20:27Z

3

Typically, this sort of thing is done either with awk or with a simple read loop.

#!/bin/bash

exec < readfile.csv || exit 1
read header # read (and ignore) the first line
while IFS=, read name id val number ip; do
    echo name = "$name"
    echo id = "$id"
    echo val = "$val"
    echo number = "$number"
    test -n "$ip" && echo ip = "$ip"
done

edited Mar 12, 2019 at 15:20

answered May 15, 2017 at 12:06

William Pursell

214k49 gold badges279 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Inian · Accepted Answer · 2017-05-15 11:54:19Z

2

You should probably use Awk for these kind of tasks which is ideal. Read more about the variables FS,OFS,FNR, NR and NF from this built-in variables in Awk page before proceeding further.

Awk in general processes one line at a time tracked by the variable NR, so skip the header line using NR>1 i.e. meaning, skipping line NR==1,

The variables FS and OFS define how your input and output lines are de-limited. Since you are working with a .csv file, it would be wise to set it to ,. Once this is done as part of BEGIN{} clause (executed before processing of the file happens), the individual fields can be printed from $1..$NF where NF is the max number of fields separated by the FS in each line.

so,

awk 'BEGIN{FS=OFS=","}NR>1{for(i=1;i<=NF;i++) print $i}' input.csv

Answer updated to match OP's requirement,

awk -v str="https://my_win_loc/personinfo" 'BEGIN{FS=OFS=","} NR>1 {print str"/"$1}' input.csv

edited May 15, 2017 at 11:54

answered May 15, 2017 at 10:25

Inian

87k15 gold badges166 silver badges183 bronze badges

10 Comments

Raji Over a year ago

Thank you Inian. On executing the above mentioned awk line, I am getting output with the header. Name Id VAl Number IP .The values I can store on new line but still header is visible

Inian Over a year ago

@Raji: Sorry it was a mistake, missed adding NR>1 in the command. Try it now

Inian Over a year ago

@Raji: This prints each element in a separate line (excluding header). How would you like to print it? Can you show an example?

Raji Over a year ago

Thanks a lot Inian. The header went away with the modified line. I want to display code as follows in new line: val1:FirstVal Val2:SecondVal

Raji Over a year ago

the above mentioned are all in new line. I am not able to edit comments to put in new line

|

Sanjuktha · Accepted Answer · 2017-05-15 10:38:24Z

0

The standard awk library can be used for that purpose. To exclude header line, you can

awk '{if (NR!=1) {print}}' readfile.csv

Here NR is the number of lines.

edited May 15, 2017 at 10:38

Sanjuktha

1,0813 gold badges13 silver badges29 bronze badges

answered May 15, 2017 at 10:18

bilalsamioguz

291 silver badge3 bronze badges

1 Comment

Raji Over a year ago

Hi Bilalsamioguz. Thank you for the response. The code works fine but displays output with comma. Inian helped in fixing the code.

Collectives™ on Stack Overflow

Read CSV file in a Shell Script till End Of Line Without Header

3 Answers 3

Comments

10 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

10 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related