Bash Script with AWK output to CSV

Question

I had amazing help on an AWK script here and thought to myself it would be really cool to have the exact same output I am monitoring on the CLI to go to a CSV file. I did research and found a great answer here, it basically showed code like this:

awk '{print $1","$2","$3","$4","$5}' < /tmp/file.txt > /tmp/file.csv

The first issue I have is /tmp/file.txt is not needed as my code is already producing the string with separated values. I don't know if my variables would work without running all new AWK commands, so I would prefer to just tag it to the end of the previous AWK command if possible. But I don't know how to implement the same concept within the actual script I am using. Could anyone show me the formatting schema I would need to tag this into the end of my script?

My ever-evolving script looks like this:

#!/bin/bash
CURRENT_DATE=`date +%Y-%m-%d`
tail -fn0 /var/log/pi-star/MMDVM-"$CURRENT_DATE".log | gawk '
match($0, /received.*voice header from ([[:alnum:]]+) to ([[:alnum:]]+ 
[0-9]+)/, a) {
in_record = 1
call_sign = a[1]
channel = a[2]
}
in_record && match($0, /DMR ID: ([0-9]+)/, a) {
dmr_id = a[1]
}
in_record && match($0, /([0-9.]+) seconds, ([0-9]+)% packet loss, BER: 
([0-9.]+)%/, a) {
in_record = 0
print call_sign, channel, dmr_id, a[1], a[2], a[3]
}
' OFS=,
done

I still want to monitor via the terminal, I just think the appended output to CSV would be the icing on the cake. Am I overthinking it? Should it just be a separate script? If so, how?

TEE looks like a very powerful command! I will research it today for sure. Thank you. — Jae Nulton
– Jae Nulton, Commented Jun 9, 2019 at 21:31
I have tried inserting TEE again with schema issues. I put TEE between the TAIL and GAWK and it outputs the tailed document perfectly to a secondary file (including all the rubbish that GAWK removes), but I have tried placing the same TEE command after GAWK in many places and it just will not work, constant syntax errors. A shame because after reading on TEE it looks like the perfect solution. :/ — Jae Nulton
– Jae Nulton, Commented Jun 9, 2019 at 23:45
wrt constant syntax errors - If you want help debugging syntax errors you should tell us what the error message are you're getting. — Ed Morton
– Ed Morton, Commented Jun 10, 2019 at 2:23
Answer updated. Sorry for not posting the exact errors, but thank you Ed. — Jae Nulton
– Jae Nulton, Commented Jun 10, 2019 at 3:06

Jae Nulton · Accepted Answer · 2019-06-10 03:06:00Z

1

After posting the question with a better description on another thread someone responded with a correct answer. He said that basically what I was seeing is awk buffering output when it's going to a pipeline (since that's lower-overhead), but writing it immediately when it's going to a TTY. He went on to offer a solution by calling fflush() from the awk program.

"Call fflush(), after your print command, add an extra command fflush()."

That fixed it. Thank you all for your efforts.

answered Jun 10, 2019 at 3:06

Jae Nulton

3912 gold badges6 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Bash Script with AWK output to CSV

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related