How to use output of a command inside an awk command?

Question

I want to print out the last update of a log file and nothing above it (old logs). Every 5 minutes the log is updated/appended to, and there is no option to overwrite instead of append. The amount of lines per update don't vary now, but I don't want to have to change the script if and when new fields are added. Each appendage starts with "Date: ...."

This is my solution so far. I'm finding the line number of the last occurrence of "Date" and then trying to send that to "awk 'NR>line_num_here filename" -

line=$(grep -n Date stats.log | tail -1 | cut --delimiter=':' --fields=1) | awk "NR>$line" file.log

However, I cannot update $line! It always holds the very first value from the very first time I ran the script. Is there a way to correctly update $line? Or are there any other ways to do this? Maybe a way to directly pipe into awk instead of making a variable?

If line always contains the same value, it means that the log file never changes. Also you should use a ; instead of | in front of awk — hek2mgl
– hek2mgl, Commented Dec 2, 2015 at 22:43
I'm thoroughly confused at this point. Line was not updating correctly. Maybe it was a typo. Thanks for the ; suggestion as well. Forgot about that. — Thisisstackoverflow
– Thisisstackoverflow, Commented Dec 2, 2015 at 22:53
Yes. However it is too complicated. awk is grep, tail and cut in once :) — hek2mgl
– hek2mgl, Commented Dec 2, 2015 at 23:40
For a newbie like me it helps to see each application do its thing, but I agree! — Thisisstackoverflow
– Thisisstackoverflow, Commented Dec 3, 2015 at 0:07

hek2mgl · Accepted Answer · 2015-12-03 00:16:25Z

1

The problem in your solution is that you need to replace the pipe in front of awk by a ;. These are two separate commands which would normally appear on two separate lines:

line=$(...)
awk -v "NR>$line" file

However, you can separate them by a ; if the should appear on the same line:

line=$(...); awk -v "NR>$line" file

But anyway you can significantly simplify the command. Simply use twice awk twice, like this:

awk -v ln="$(awk '/Date/{l=NR}END{print l}' a.log)" 'NR>ln' a.log

I'm using

awk '/Date/{l=NR}END{print l}' a.log

to obtain the line number of the last occurrence of Date. This value get's passed via -v ln=... to the outer awk command.

edited Dec 3, 2015 at 0:16

answered Dec 2, 2015 at 22:48

hek2mgl

159k31 gold badges263 silver badges279 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Thisisstackoverflow Over a year ago

Much sleeker. Thanks!

Tom Fenech Over a year ago

Well done for working out what the OP is trying to do - it took me a while! I guess that another, possibly more common, option would be to read the file twice with a single invocation of awk.

Tom Fenech · Accepted Answer · 2015-12-02 23:16:13Z

0

Here's a way you could do it, in one invocation of awk and only reading the file once:

awk '/Date/ { n = 1 } { a[n++] = $0 } END { for (i = 1; i < n; ++i) print a[i] }' file

This writes each line to an array a, resetting the counter n back to 1 every time the pattern /Date/ matches. It then loops through the array once the file has been read, printing all the most recently saved values.

answered Dec 2, 2015 at 23:16

Tom Fenech

75.1k13 gold badges119 silver badges154 bronze badges

Collectives™ on Stack Overflow

How to use output of a command inside an awk command?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related