I have a situation in awk where I need to convert an input format into another format and later use the number of records processed separately. Is there any way I can use a shell variable to get the value of NR in the END section? Something like:
cat file1 | awk 'some processing END{SHELL_VARIABLE=NR}' > file2
Then later use SHELL_VARIABLE outside awk.
I do not want to process the file and then do a wc -l separately as the files are huge.
One way: Use the redirection inside your awk command and print your result in the END block. And use command substitution to read the result in a shell variable:
my_var=$(awk '{ some processing; print "your output" >>file2 } END { print NR }' file1)
No subprocess can affect the parent's environment variables. What you can do is have awk write output to the file directly, then have it print the value you want to stdout and capture it. Or if you prefer, you could reverse that and have awk just print it to a file and read it back afterwards.
Incidentally, you have a UUOC.
rows=$(awk '{ ...; print > "file2"} END {print NR}' file1)
Or
awk '... END{print NR > "rows"}' file1 >file2
rows=$(<rows)
rm rows
awk: i/o error occurred on numlines