awk output csv file format syntax

Question

I am trying to extract data from multiple text files using awk.

I am hitting a road block on trying to obtain the output file I desire.

I have tried many instances of using OFS and ORS to no avail...

The input text files are always of the same structure, fields do not change, only the data values.

text input files filename example: FCS0702_N4_S2_AAI135H_IORevInf.txt

text input files example:

I/O Module:
Line : 1 Node : 4 Slot : 2 Type : AAI135H
0000    0000 2000 0000 C000  C000 8000 4400 AAA0    .. .........D...
0010    AA55 000F 4CCC 1000  0FF4 0000 0E01 0E01    .U..L...........
0020    0000 0000 0000 0000  0000 0000 0000 0000    ................
0030    00D9 0000 0000 0000  0000 0000 0000 0000    ................
0040    0010 0D00 0814 2000  0B00 03DC 0088 0000    ...... .........
0050    0000 0100 1010 0000  0000 4010 0000 0160    ..........@....`
0060    0000 0000 0400 8008  0000 0000 8008 0000    ................
0070    0000 0000 0000 0000  0000 0000 0000 0000    ................
0080    0000 0000 0000 0000  0B10 0B5C 0B74 0BB0    ...........\.t..
0090    0BC8 0BFC 0C3C 0000  A5D7 003C 4141 4931    .....<.....<AAI1
00A0    3335 2D48 3030 2020  5332 5532 4830 302D    35-H00  S2U2H00-
00B0    4631 2D2D 4333 4743  3330 3733 3347 2020    F1--C3GC30733G

CSV output file desired:

FCS0702_N4_S2_AAI135H_IORevInf.txt,Line1,Node4,Slot2,TypeAAI135H,AAI135-H00S2,U2H00-F1--,C3GC30733G

current awk script:

#!/bin/sh
touch output.csv
for inputfile in *.txt
do
echo $inputfile >>output.csv
### awk 'NR==2 {printf $0}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,1,4)}'  "$inputfile" >>output.csv
awk NR==2 {printf substr($0,8,1)}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,10,4)}' "$inputfile" >>output.csv
awk  NR==2 {printf substr($0,17,1)}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,19,4)}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,26,1)}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,28,4)}' "$inputfile" >>output.csv
awk NR==2 {printf substr($0,35,7)}' "$inputfile" >>output.csv
awk NR==12 {printf substr($0,60,4)}' "$inputfile" >>output.csv
awk NR==13 {printf substr($0,48,6)}' "$inputfile" >>output.csv
awk NR==13 {printf substr($0,56,7)}' "$inputfile" >>output.csv
awk NR==14 {print substr($0,48,16)}' "$inputfile" >>output.csv
done

As previously stated, I have tried many attempts of OFS and ORS, to no avail...

    awk 'BEGIN{ORS=","} NR==2 {printf substr($0,1,4)}'  "$inputfile" >>output.csv

doesn't work.

    awk -v ORS="," NR==2 {printf substr($0,1,4)}'  "$inputfile" >>output.csv

doesn't work.

I have tried many others that I have forgotten about.

Searching via google doesn't really help, nor does the documentation found.

I guess I need an example, sorry, really hung up on this, for some reason, sigh...

kamae · Accepted Answer · 2014-06-18 15:02:30Z

1

Following command will output your desired output.

awk '
BEGIN {printf "%s,", "'"$inputfile"'"}
NR==2 { gsub(/ : /, ""); gsub(/ /, ","); printf "%s,", $0 }
NR==12 { printf "%s", substr($0,60,4) }
NR==13 { printf "%s,%s", substr($0,47,6), substr($0,56,7) }
NR==14 { printf "%s,%s\n", substr($0,47,4), substr($0, 51,12) }
' $inputfile

The function printf takes format as 1st argument. You should avoid to pass input data as 1st argument, since it might contain conversion specifiers.

answered Jun 18, 2014 at 15:02

kamae

1,8771 gold badge17 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user3749927 Over a year ago

Wow! This is soo close! The comma is off by one field: FCS0105_N5_S1_AAI135H_IORevInf.txt,Line1,Node5,Slot1,TypeAAI135H,AAI135-H00,S2U2H00F1--,-C3GF21088G Should be: FCS0105_N5_S1_AAI135H_IORevInf.txt,Line1,Node5,Slot1,TypeAAI135H,AAI135-H00,S2U2H00F1---,C3GF21088G I really have through go through this. I very much want to learn this.

user3749927 Over a year ago

Oh, I did change: NR==13 { printf "%s,%s", substr($0,47,6), substr($0,56,7) } To: NR==13 { printf "%s,%s", substr($0,48,6), substr($0,56,7) }

Etan Reisner · Accepted Answer · 2014-06-18 15:01:51Z

OFS controls what goes between fields in $0 (when it has recalculated it from modified fields) when awk prints it and what goes between individual arguments to print. If you explicitly print a value, and only one value, then OFS never comes into play.

ORS on the other hand controls what awk prints at the end of the line output by print.

Notice how neither of those descriptions above talked about printf? That's because printf doesn't do either of those things.

First things first you should combine those dozen or so awk invocations into a single awk script.

Then you need to start either replacing fields in the line directly (by modifying $1 directly or modifying $0 and using $1=$1 or similar to force awk to recalculate $0) and then using print or you need to start using print substr(...),substr(...).

Collectives™ on Stack Overflow

awk output csv file format syntax

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related