Grep from variable stored in a csv file

Question

I'm try to grep for everything within "test.csv" from the contents on book.1

while IFS= read -r result
do
grep -r $result test.csv >> output.csv

echo $result

done<book1.csv

My book 1 has 1 row of data is column A.

I am getting the results of book1.csv shown on the screen so it is parsing correctly, but I'm getting nothing in the output.csv

I have manually checked the grep & there is contents of book1.csv within the test.csv

What have I done wrong?

EDIT

SAMPLE Input book1.csv

HOSTNAME
TEST1
TEST2

Sample input test.csv

blank,TEST1.abc.123,blank,date

I want to get every line from book1.csv that is in test.csv

Cat -a book1 result

TEST1^M$

when i do this "grep -r TEST1 test.csv >> output.csv" without a script i get the expected data so its something to do with my $result variable — Bunion
– Bunion, Commented Sep 15, 2017 at 9:00
grep -Ff book1.csv test.csv would usually work but you have to provide sample data from test.csv as well. Also check output of cat -A book1.csv and cat -A test.csv to make sure you don't have DOS carriage returns. — anubhava
– anubhava, Commented Sep 15, 2017 at 9:00
sample imput added, when i do the above grep -Ff i get nothing. However if i do grep TEST1 test.csv i get results so im not sure what is wrong — Bunion
– Bunion, Commented Sep 15, 2017 at 9:14

anubhava · Accepted Answer · 2017-09-15 09:21:43Z

2

As suspected initially your .csv files have DOS line ending with \r characters, which is causing grep -f to not match since Unix files just end with \n.

You can use this command with tr that removes \r from both .csv files before running grep:

grep -Ff <(tr -d '\r' < book1.csv) <(tr -d '\r' < test.csv)

blank,TEST1.abc.123,blank,date

tr -d '\r' < book1.csv will delete \r or carriage return character from given files.

answered Sep 15, 2017 at 9:21

anubhava

790k67 gold badges603 silver badges671 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Charles · Accepted Answer · 2017-09-15 09:29:38Z

1

using fgrep should work. With the data that you provided, it gives:

fgrep -f book1.csv test.csv

Output:

blank,TEST1.abc.123,blank,date

fgrep is strictly equivalent to grep -F

from grep man page :

Matcher Selection
   -F, --fixed-strings
          Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.  (-F is specified by POSIX.)


Matching Control
   -f FILE, --file=FILE
          Obtain  patterns  from FILE, one per line.  The empty file contains zero patterns, and therefore matches nothing.  (-f is specified
          by POSIX.)

As pointed out by anubhava , you need to remove the DOS character. To do that, just use the dos2unix command:

fgrep -f <(dos2unix book1.csv) <(dos2unix test.csv)

edited Sep 15, 2017 at 9:29

answered Sep 15, 2017 at 9:22

Charles

814 bronze badges

Collectives™ on Stack Overflow

Grep from variable stored in a csv file

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related