processing text using awk

Question

I am trying to process a csv file using awk. I have five rows in my csv file. I want my column 1 & column 3 combined to be unique. Using awk I wrote

awk -F "," '{print $1,",",$2,","$3,",",$4,",",$5}' A.csv|sort|uniq >B.csv

What this code does is to have a unique row with 1 to 5 rows combined. I want my other columns to be in final csv file but sorted and uniq only by 1st and 3rd columns.

When you say combined based on a unique key does that mean the new combined row could have 6 elements in addition to the keys? If so, sounds like you picked the wrong tool. A Perl script using a hash table would work out nicely in this case. — tawman
– tawman, Commented Jan 15, 2011 at 1:09
@tawman awk supports associative arrays just fine. But until the OP actually provides something concrete that we can see, I doubt anybody is going to expend the time writing something up when we can only speculate what the OP really wants. — SiegeX
– SiegeX, Commented Jan 15, 2011 at 1:14
Agreed, awk has quite an in-depth language that rarely the unix user dives into that far. I did once a long time ago, but easier to use the tools you know. I did not mean to insult the power of awk! — tawman
– tawman, Commented Jan 15, 2011 at 1:17
You don't have to use all those quoted commas, by the way. awk -F, '{print $1,$2,$3}' OFS=, filename — Dennis Williamson
– Dennis Williamson, Commented Jan 15, 2011 at 3:49

ghostdog74 · Accepted Answer · 2011-01-15 01:31:47Z

2

assuming you want to keep at least one of the duplicate lines

$ more file
1 2 3 4 5 6
7 4 6 3 5 8
1 43 3 6 5 10
2 3 4 4 4 4

$ awk '(!($1$3 in u)){u[$1$3]=$0}END{for(i in u ) print u[i]}' file
7 4 6 3 5 8
1 2 3 4 5 6
2 3 4 4 4 4

If you don't want to keep duplicate lines

$ awk '($1$3 in u){delete u[$1$3];next}{u[$1$3]=$0}END{for(i in u ) print u[i]}' file
7 4 6 3 5 8
2 3 4 4 4 4

answered Jan 15, 2011 at 1:31

ghostdog74

346k62 gold badges264 silver badges349 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Vijay Over a year ago

@ghostdog...could you please explein the solution here

Collectives™ on Stack Overflow

processing text using awk

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related