Parsing Text File using array and Sed and Awk in Bash

Question

I have what appears to me a complex text file that include around 300 entries. I have no idea how to go about parsing this file to get the output I want. Each of my network users has an entry in the file. So in the text file, each user name starts with:

USER:martha
USER:Othello
USER:darwin

Underneath each user entry in the file there are a host of information I require , but one user can have one entry and another can have multiple entries. Here is the example of 3 such entries

USER:martha
    POSITION: 170.198.82.13 [VLT(304394),PT(FULL)]
            CLIENT: jcrm19.1.p2ps -258-
            ACCESSPOINT: 170.198.82.13/net
            APPLICATION: 91

USER:othello 
    POSITION: 170.198.80.212 [VLT(307571),PT(FULL)]
            CLIENT: jcrm15.1.p2ps -258-
            ACCESSPOINT: 170.198.80.212/net
            APPLICATION: 256

            CLIENT: jcrm15.1.p2ps -258-
            ACCESSPOINT: 170.198.80.212/net
            APPLICATION: 256

    POSITION: 170.198.80.209 [VLT(306561),PT(FULL)]
            CLIENT: jcrm14.1.p2ps -258-
            ACCESSPOINT: 170.198.80.209/net
            APPLICATION: 256

            CLIENT: pwrm14.1.p2ps -258-
            ACCESSPOINT: 170.198.80.209/net
            APPLICATION: 256

            CLIENT: pwrm14.1.p2ps -258-
            ACCESSPOINT: 170.198.80.209/net
            APPLICATION: 256


USER:darwin
    POSITION: 170.198.19.102 [VLT(297987),PT(FULL)]
            CLIENT: jcrm16.1.p2ps -258-
            ACCESSPOINT: 170.198.19.102/net
            APPLICATION: 91

The final output should look as follow:

USER        Position           Client     Application 

Martha      170.198.82.13       jcrm19      91
Othello     170.198.80.212      jcrm15      256
Othello     170.198.80.209      jcrm14      256
Martin      170.198.19.102      jcrm16      91

I have some experience with arrays and I could grep out some of the information and assign to variable and print them. But I just don't know how to read the information into the arrays as the entries under each "USER" since they are of different length and content.

So How do I read USER: martha and then jump to user:othello ? Also, under user:othello there are two "Positions" that I need to grab. I just don't know how to put the content I'm looking for into array variables or regular variables. I never had to parse a file that had different length and content data for each use. Not sure how many lines I have to read before I start reading and assigning values to array or values for the next user> Can you provide some hints or perhaps a piece of code that I can start with ?

Thanks

anubhava · Accepted Answer · 2014-05-13 20:27:18Z

2

Using awk with column:

awk -F '[: ]+' 'BEGIN{print "USER", "Position", "Client", "Application"} 
  $1=="USER"{u=$2} $2=="POSITION"{p=$3}$2=="CLIENT"{c=$3}
  $2=="APPLICATION"&&p{print u, p, c, $3; p=""}' file | column -t

USER     Position        Client         Application
martha   170.198.82.13   jcrm19.1.p2ps  91
othello  170.198.80.212  jcrm15.1.p2ps  256
othello  170.198.80.209  jcrm14.1.p2ps  256
darwin   170.198.19.102  jcrm16.1.p2ps  91

answered May 13, 2014 at 20:27

anubhava

790k67 gold badges603 silver badges671 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Etan Reisner Over a year ago

This assumes that the fields will always be in the order in the sample. Which may be correct but should be pointed out.

theuniverseisflat Over a year ago

hi thx .. I tried this. Yes the fields are always in this order. It works with the sample I provided and not with the text file I have .. Not sure why . I have to look at the log file again. Thx for quick response. will have to look later as I have to run to catch a train thx again!! anubhava

anubhava Over a year ago

Sure let me know if you need further assistance.

Mark Setchell · Accepted Answer · 2014-05-13 20:30:58Z

0

I haven't got my Mac to hand so this is untested...

awk -F: '/^USER:/{u=$2} /POSITION:/{p=$2} /CLIENT:/{c=$2} /APPLICATION:/{print u,p,c,$2}' yourfile

answered May 13, 2014 at 20:30

Mark Setchell

210k32 gold badges309 silver badges503 bronze badges

3 Comments

Kent Over a year ago

OP just wants to pick the block with POSITION, yours prints entry for each block

Mark Setchell Over a year ago

@Kent Oh, yes, you are right, thank you. I'll leave him to choose between your and anubhava's answers.

theuniverseisflat Over a year ago

Hi Mark. I tried your one liner and worked great. Not sure how it works , but it works. So thank you. I really have to understand how this position stuff work. also when I ran it against my original log file, it produced an additional columns of data .. but that's fine./

Kent · Accepted Answer · 2014-05-13 20:43:16Z

0

awk -v RS="" -F'[:\n ]*' '/^USER/{u=$2}
 /POSI/{p=/^USER/?$4:$3
 for(i=1;i<=NF;i++)
     if($i=="CLIENT"){sub(/\..*/,"",$(i+1))
                      print u,p,$(i+1),$NF;break}}' file

the output without header:

martha 170.198.82.13 jcrm19 91
othello 170.198.80.212 jcrm15 256
othello 170.198.80.209 jcrm14 256
darwin 170.198.19.102 jcrm16 91

you could add header and pipe to column -t to gain better format

answered May 13, 2014 at 20:43

Kent

197k36 gold badges248 silver badges317 bronze badges

1 Comment

theuniverseisflat Over a year ago

Hi Kent. Thanks for reply. Sorry I never got around this post. I tried ur code and it does not do anything. It produces no error and it does not produces any output?

Collectives™ on Stack Overflow

Parsing Text File using array and Sed and Awk in Bash

3 Answers 3

3 Comments

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related