Extracting text from file using bash

Question

I am new to Linux and have a very large text log file from which to extract. I thought to use bash?

For example, the file contains:

Node:xyz
Time:01/07/13 14:26:17
INFO: Trusted certif ok

Node:abc
Time:01/07/13 14:26:18
INFO: Trusted certif ok

Node:def
Time:01/07/13 14:26:18
INFO: Trusted certif not ok

I need to extract the text after Node: and add it to the text after Info: to display on one line, output to be redirected to a new file. I am trying awk and sed, but not figured it out yet. Help much appreciated.

Example output would look like:

xyz Trusted certif ok
abc Trusted certif ok
dbf Trusted certif not ok

Gilles Quénot · Accepted Answer · 2013-05-28 07:42:19Z

13

Try doing this :

in awk

awk -F: '/^Node/{v=$2}/^INFO/{print v $2}' file.txt

in bash :

while IFS=: read -r c1 c2; do
    [[ $c1 == Node ]] && var=$c1
    [[ $c1 == INFO ]] && echo "$var$c2"
done < file.txt

in perl :

perl -F: -lane '
    $v = $F[1] if $F[0] eq "Node";
    print $v, $F[1] if $F[0] eq "INFO"
' file.txt

in python (in a file, Usage : ./script.py file.txt ):

import sys
file = open(sys.argv[1])
while 1:
    line = file.readline()
    tpl = line.split(":")
    if tpl[0] == "Node":
        var = tpl[0]
    if tpl[0] == "INFO":
        print var, tpl[1]
    if not line:
        break

edited May 28, 2013 at 7:42

answered Jan 7, 2013 at 22:14

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Allen Over a year ago

Thank you all very much. awk is awesome, as is your help.

TrueY Over a year ago

@EdMorton: what is the problem with the pure bash solution?

Ed Morton Over a year ago

@TrueY shell is an environment from which to call tools. It has programming language constructs (loops, etc.) to help you sequence the order in which you call tools. It is not a tool for parsing text files and so it's capabilities for doing that are extremely limited and it's side-effects non-obvious. For example, courtesy of the missing -r argument for read the script you posted will incorrectly interpret backslashes, and the use of echo will only work on some systems with some inputs. There may be additional edge cases it fails for and it's over twice the length of the robust awk script.

TrueY Over a year ago

@EdMorton: Thx 4 the answer! Generaly you are right. Every shell, interpreter and compiler has its own strength and weakness. The specific case will decide which tool to use. IMHO if some tool os not needed, it is better not to use. So in this case I prefer the pure bash version.

Gilles Quénot Over a year ago

@canfiese: try gawk

|

perreal · Accepted Answer · 2013-01-07 22:17:35Z

2

Using sed:

sed -n '/^Node/N;/Time/N;s/^Node:\([^\n]*\)\n[^\n]*\n[^ ]* /\1 /p' input

answered Jan 7, 2013 at 22:17

perreal

98.7k23 gold badges159 silver badges187 bronze badges

Comments

Vijay · Accepted Answer · 2013-01-08 11:06:22Z

0

perl -F: -lane '$x=$F[1] if(/^Node:/);if(/^INFO:/){print "$x".$F[1];}' your_file

tested below:

> cat temp
Node:xyz
Time:01/07/13 14:26:17
INFO: Trusted certif ok

Node:abc
Time:01/07/13 14:26:18
INFO: Trusted certif ok

Node:def
Time:01/07/13 14:26:18
INFO: Trusted certif not ok

> perl -F: -lane '$x=$F[1] if(/^Node:/);if(/^INFO:/){print "$x".$F[1];}' temp
xyz  Trusted certif ok
abc  Trusted certif ok
def  Trusted certif not ok

answered Jan 8, 2013 at 11:06

Vijay

67.7k94 gold badges238 silver badges327 bronze badges

Comments

Sidharth C. Nadhan · Accepted Answer · 2013-05-28 09:04:32Z

0

sed -n 'N;N;s/\n.*\n/ /;s/\S*://g;p;n' file

edited May 28, 2013 at 9:04

answered May 28, 2013 at 8:46

Sidharth C. Nadhan

2,2832 gold badges18 silver badges18 bronze badges

Collectives™ on Stack Overflow

Extracting text from file using bash

4 Answers 4

9 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

9 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related