Redirect output to specific column in file

Question

Daily I am running approx 100 queries in a script that each produce a single value output (a number). I'm currently outputting each query's value to multiple files, appending each days' output to the bottom of the query's respective file:

query1 goes to file1
date value
Jan-01 35
Jan-02 74
query2 goes to file2
date value
Jan-01 834
Jan-02 342
etc

What I would like is to build a single file that contains a column for each query and append the results accordingly:

Date Query1 Query2 etc
Jan01 35 834 etc
Jan02 74 342 etc

My current query:

            query `yesterday` direct | awk '$1=="FA"{count++}END{print count}' > /x/2018/files/"$file1"
            query `yesterday` direct | awk '$1=="FM"{count++}END{print count}' > /x/2018/files/"$file2"
            query `yesterday` direct | awk '$1=="FT"{count++}END{print count}' > /x/2018/files/"$file3"

In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm. — seshoumara
– seshoumara, Commented Apr 4, 2018 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query: — Steve
– Steve, Commented Apr 4, 2018 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file? — seshoumara
– seshoumara, Commented Apr 4, 2018 at 13:26
Or simply (queryDate; query1; query2; query3 ...) | xargs >> master_file perhaps? — seshoumara
– seshoumara, Commented Apr 4, 2018 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc — Steve
– Steve, Commented Apr 4, 2018 at 14:06

seshoumara · Accepted Answer · 2018-04-04 03:59:42Z

0

Assuming the OP confirms the behavior I mentioned in the comments, this script should work:

awk '
     FNR != 1 { sub("-", "", $1); dates[$1] = dates[$1] FS $2 }
     END {
         for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
         for(d in dates) print d dates[d]
     }
' $(cat "list.txt")

I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.

answered Apr 4, 2018 at 3:59

seshoumara

8725 silver badges7 bronze badges

Add a comment |

Stack Exchange Network

Redirect output to specific column in file

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Redirect output to specific column in file

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions