shell parse csv file, assign fields to variable and print it in multiple columns

Question

Im trying to curl a csv file and parse it based on their property and print it back by using variable names.

File 1:

10.0.0.1,gateway,name_of_device
10.2.4.5,server,name_of_device
10.3.5.6,PC,name_of_device

My script below,

#!/bin/sh

input=$(curl http://example.com/1.txt)

ip=$(echo "$input" | awk -F ',' '{print $1}')
type=$(echo "$input" | awk -F ',' '{print $2}')
name=$(echo "$input" | awk -F ',' '{print $3}')

echo "$ip $type $name"

This prints,

10.0.0.1
10.2.4.5
10.3.5.6
gateway
server
PC
name_of_device
name_of_device
name_of_device

but the expected output should be like,

10.0.0.1 gateway name_of_device
10.2.4.5 server name_of_device
10.3.5.6 PC name_of_device

Tried different options like,

assigning output to another variable and echo'ing it,

# output="$source_ip $tag_text"
# echo -e $output

printf statements:

# printf "%-20s | %-20s" "$source_ip" "$tag_text"


# printf "$source_ip" "$tag_text"

William Pursell · Accepted Answer · 2016-09-02 12:27:04Z

2

awk is certainly the correct tool to use for this, but you can also do:

curl http://example.com/1.txt |
while IFS=, read ip type name rest; do
  echo $ip $type $name
done

The variables will lose their value after the while loop finishes, since it is in a sub-shell.

answered Sep 2, 2016 at 12:27

William Pursell

214k49 gold badges279 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

fedorqui Over a year ago

... and if you want to have the variables in the main shell, use process substitution with while ... done < <(curl ...)

fedorqui · Accepted Answer · 2016-09-02 12:07:39Z

2

Just set the input field separator to the comma and let awk handle the rest of it:

$ awk -F, '$1=$1' file
10.0.0.1 gateway name_of_device
10.2.4.5 server name_of_device
10.3.5.6 PC name_of_device

By saying $1=$1, awk recalculates the fields and sets OFS all around, which makes all commas (FS) to be replaced with spaces (OFS).

edited Sep 2, 2016 at 12:07

answered Sep 2, 2016 at 12:02

fedorqui

293k113 gold badges592 silver badges640 bronze badges

10 Comments

James Brown Over a year ago

You didn't $1=$1. That causes awk to rebuild the record and actually utilize the OFS.

Karthik Over a year ago

yes, this works. But how can i parse as rows? if I call an IP (first column), it should provide appropriate type and name (which is 2nd and 3rd column).

Aaron Over a year ago

@Karthik How do you "call an IP" ? Maybe using grep to retrieve the line with the specified IP is what you want?

Karthik Over a year ago

@Aaron may be yes.

Karthik Over a year ago

@fedorqui yes, understood. My final goal is to push these data into some db for other purposes. I really haven't started looking into db importing and it was too early to post that question here. So the db would contain a defied set of columns and that why i have to ignore them inside this script itself. Your suggestions was very helpful. thanks

|

Collectives™ on Stack Overflow

shell parse csv file, assign fields to variable and print it in multiple columns

2 Answers 2

1 Comment

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related