How can I use array with awk in Linux

Question

INFO #my-service# #add# id=67986324423 isTrial=true
INFO #my-service# #add# id=43536343643 isTrial=false
INFO #my-service# #add# id=43634636365 isTrial=true
INFO #my-service# #add# id=67986324423 isTrial=true
INFO #my-service# #delete# id=43634636365 isTrial=true
INFO #my-service# #delete# id=56543435355 isTrial=false

I want to count the lines which are having unique ids with #add# attribute in them & having isTrial=true.

This is my current solution and I want to know why my array is not printing

BEGIN { print "Begin Processing of various Records"}

{if($3~"add" && $5~"true")
   {
   ++i; 
   if($4 not in arr){arr[i]=$4;++j} 
   }
  {print $0}
}

 END {print "Process Complete:--------"j}

Steve · Accepted Answer · 2013-01-11 04:46:28Z

1

You need to test to see if the fourth field is not already in the array, like so:

BEGIN {
    print "Begin Processing of various Records"
}

$3 ~ /add/ && $5 ~ /true/ && !a[$4]++ {

    i++
    print
}

END {
    print "Process Complete. Records found:", i
}

Results:

Begin Processing of various Records
INFO #my-service# #add# id=67986324423 isTrial=true
INFO #my-service# #add# id=43634636365 isTrial=true
Process Complete. Records found: 2

Here's some info that may interest you. HTH.

As per comments below, you could also do this:

BEGIN {
    print "Begin Processing of various Records"
}

$3 ~ /add/ && $5 ~ /true/ && !a[$4] {

    a[$4]++
    print 
}

END {
    print "Process Complete. Records found:", length(a)
}

Note that this is very different from:

BEGIN {
    print "Begin Processing of various Records"
}

$3 ~ /add/ && $5 ~ /true/ && !a[$4] {

    # See the line below. I may not have made it clear in the comments that
    # you can indeed add things to an array without assigning the key a
    # value. However, in this case, this line of code will fail because our
    # test above (!a[$4]) is testing for an absence of value associated
    # with that key. And the line below is never assigning a value to the key!
    # So it just won't work.

    a[$4]


    # Technically, you don't need to increment the value of the key, this would
    # also work, if you uncomment the line:

    # a[$1]=1

    print 
}

END {
    print "Process Complete. Records found:", length(a)
}

edited Jan 11, 2013 at 4:46

answered Jan 11, 2013 at 2:48

Steve

55.1k13 gold badges94 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

user175386049 Over a year ago

Can you please explain what does this means !a[$4]++

Steve Over a year ago

Sure. It simply means: 'If field 4 is not (!) in an array (called a), add it to the array, incrementing the key's value by one (++)'. Does that make sense? The fourth field is the key to the array.

user175386049 Over a year ago

I didn't get the ++ part , i didn't get what part is adding the $4 to array and what part is incrementing the key

Steve Over a year ago

So, $4 is the key. And a[$4] would be the value of that key. As soon as you write a[$4]++ the key is 'put' into the array and assigned a value of one. We use ++ to increment the value of the key, so that each time the key is found, the key's value is increased by one. You may also find this a worthwhile read. Please let me know if more explanation is needed. Cheers!

user175386049 Over a year ago

I still didn't get it. why we need to increment the value and for what purpose we are using that value. what diff it makes if value does not increment. All we are concerned is the if key is in array or not. we have no intial value of a['id=3213313']. now by incrementing you mean a['id=313123' + 1] or a['id='313123'] = 0 + 1

|

perreal · Accepted Answer · 2013-01-11 02:28:02Z

1

grep '#add#.*isTrial=true' input | sed 's/[^=]*=\([^ ]*\).*/\1/' | sort | uniq -c

answered Jan 11, 2013 at 2:28

perreal

98.7k23 gold badges159 silver badges187 bronze badges

1 Comment

user175386049 Over a year ago

Actually i am learning awk , so i would like awk solution to learn. thanks for that

Guru · Accepted Answer · 2013-01-11 02:39:30Z

1

One way using awk:

$ awk '$3 ~ /add/ && $5 ~ /true/{sub(/.*=/,"",$4);a[$4]++;}END{for (i in a)print i, a[i];}' file
43634636365 1
67986324423 2

Regarding your solution:

When you use the contains(~) operator, the pattern should always be provided in slashes(//), not directly in double quotes.
When you check $4 not in arr, it checks for $4 in the array keys, whereas you are populating $4 as an array value arr[i]=$4.

edited Jan 11, 2013 at 2:39

answered Jan 11, 2013 at 2:32

Guru

17.1k2 gold badges37 silver badges47 bronze badges

1 Comment

user175386049 Over a year ago

Can you correct my script , so that i see how far i was from the solution

Vijay · Accepted Answer · 2013-01-11 07:46:29Z

0

awk '$5~/isTrial=true/ && $3~/#add#/{a[$4]}END{for(i in a){count++}print count}'

tested here

answered Jan 11, 2013 at 7:46

Vijay

67.7k94 gold badges238 silver badges327 bronze badges

Collectives™ on Stack Overflow

How can I use array with awk in Linux

4 Answers 4

10 Comments

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

10 Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related