How to match & then replace multiple lines in shell script

Question

Hi I have this text file.

Physical interface: ge-0/0/3, Enabled, Physical link is Up
  Interface index: 132, SNMP ifIndex: 504
  Description: # SURVEILLANCE CAMERA #
  Link-level type: Flexible-Ethernet, Media type: Copper, MTU: 9000,
  LAN-PHY mode, Link-mode: Full-duplex, Speed: 1000mbps, BPDU Error: None,
   .....few more lines
Physical interface: ge-0/1/0, Enabled, Physical link is Down
  Interface index: 133, SNMP ifIndex: 505
  Link-level type: Ethernet, Media type: Fiber, MTU: 1514, LAN-PHY mode,
  Speed: 1000mbps, BPDU Error: None, MAC-REWRITE Error: None,
    .....few more lines
Physical interface: ge-0/1/3, Enabled, Physical link is Up
  Interface index: 136, SNMP ifIndex: 508
  Description: # TO CSS_I-TN-CHNN-ENB-I099 #
  Link-level type: Flexible-Ethernet, Media type: Fiber, MTU: 8000,
  LAN-PHY mode, Speed: 1000mbps, BPDU Error: None, MAC-REWRITE Error: None,
   ... few more lines

 and so on....

Now If Physical link is Up & value of MTU is 9000 then only I need to replace both the corresponding lines as.

<Pass>Physical interface: ge-0/0/3, Enabled, Physical link is Up
&
<Pass>Link-level type: Flexible-Ethernet, Media type: Fiber, MTU: 9000,

In every other situation it will be <Fail> in place of <Pass> . these values lie in different line that why I am not getting any idea of using sed or anything else..please help... here is the expected output..

 <Pass>Physical interface: ge-0/0/3, Enabled, Physical link is Up
      Interface index: 132, SNMP ifIndex: 504
      Description: # SURVEILLANCE CAMERA #
      <Pass>Link-level type: Flexible-Ethernet, Media type: Copper, MTU: 9000,
      LAN-PHY mode, Link-mode: Full-duplex, Speed: 1000mbps, BPDU Error: None,
         .....few more lines
  <Fail>Physical interface: ge-0/1/0, Enabled, Physical link is Down
     Interface index: 133, SNMP ifIndex: 505
     <Fail>Link-level type: Ethernet, Media type: Fiber, MTU: 1514, LAN-PHY mode,
     Speed: 1000mbps, BPDU Error: None, MAC-REWRITE Error: None,
            .....few more lines
   <Fail>Physical interface: ge-0/1/3, Enabled, Physical link is Up
     Interface index: 136, SNMP ifIndex: 508
     Description: # TO CSS_I-TN-CHNN-ENB-I099 #
     <Fail>Link-level type: Flexible-Ethernet, Media type: Fiber, MTU: 8000,
     LAN-PHY mode, Speed: 1000mbps, BPDU Error: None, MAC-REWRITE Error: None,
           ... few more lines

         and so on....

Did you already try something, maybe using sed? If yes, please, show us! — F. Hauri - Give Up GitHub
– F. Hauri - Give Up GitHub, Commented Apr 6, 2015 at 12:07
@AvinashRaj I have append the expected output in the original query.. — Ankur
– Ankur, Commented Apr 6, 2015 at 12:21
@F.Hauri I was trying to find any option with sed which just searches the previous occurrence of "Physical link is Up" once it searches "MTU: 9000" & replaces both of them.. but didn't succeed.. — Ankur
– Ankur, Commented Apr 6, 2015 at 12:21
what above awk? did you want the preserve the leading spaces? — Avinash Raj
– Avinash Raj, Commented Apr 6, 2015 at 12:29

Wintermute · Accepted Answer · 2015-04-06 21:19:33Z

3

With sed:

sed '/Physical link is/ { :a /MTU:/! { N; ba; }; /Physical link is Up.*MTU: 9000/ { s/\(.*\n\)\s*/<Pass>\1<Pass>/; b; }; s/\(.*\n\)\s*/<Fail>\1<Fail>/; }' filename

That is:

/Physical link is/ {                       # Block start found
  :a
  /MTU:/! {                                # fetch lines until we find the MTU
    N
    ba
  }
  /Physical link is Up.*MTU: 9000/ {       # If link is up and MTU 9000
    s/\(.*\n\)[[:space:]]*/<Pass>\1<Pass>/ # insert Pass markers
    b
                                           # we're done.
  }
  s/\(.*\n\)[[:space:]]*/<Fail>\1<Fail>/   # otherwise insert Fail markers
}

Note that with BSD sed, this cannot be used as a one-liner because of the b instructions. In that case, put the expanded (without comments, for BSD sed is easily confused) code in a file, say foo.sed, and use sed -f foo.sed filename. I've already replaced the other GNU-ism (\s) with its POSIX equivalent ([[:space:]]) there.

To keep the whitespaces at the beginning of the MTU line, remove the \s or [[:space:]]. To place the whitespaces before the result marker, put the \s or [[:space:]] inside the capturing group (i.e., $.*\n\s*$).

Also note: This assumes that every interface description has an MTU field.

Alternatively, you might try this awk script:

awk -v RS='Physical interface:' -F '\n' -v OFS='\n' '{ result = "<Fail>" } /Physical link is Up/ && /MTU: 9000/ { result = "<Pass>" } NR != 1 { for(i = 1; i <= NF; ++i) { if(index($i, "MTU:")) { sub(/^ */, result, $i) } } print result RS $0 }' filename

This splits the file into records at Physical interface: and the records into fields at newlines. Then:

{ result = "<Fail>" }                  # result is Fail 
/Physical link is Up/ && /MTU: 9000/ { # unless link is up and MTU 9000
  result = "<Pass>"
}

NR != 1 {                              # the first record is the empty string
                                       # before the first actual record, so
                                       # we remove it.
  for(i = 1; i <= NF; ++i) {           # wade through the fields (lines)
    if(index($i, "MTU:")) {            # find the MTU line
      sub(/^ */, result, $i)           # put the marker there. To keep the
                                       # whitespace, use $i = result $i
                                       # instead, or sub(/^ */, "&" result, $i)
                                       # to keep the spaces before the marker.
    }
  }
  print result RS $0                   # once done, print the whole shebang.
                                       # We have to reinsert the record
                                       # separator because it was removed
                                       # by the splitting.
}

Note that a multi-character RS is not strictly POSIX-conforming. The most common awks (gawk and mawk) support it, though. Notably, BSD awk does not.

edited Apr 6, 2015 at 21:19

answered Apr 6, 2015 at 12:42

Wintermute

44.3k5 gold badges85 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

mklement0 Over a year ago

Good stuff. The sed solution doesn't preserve the whitespace at the start of the Link-level type lines. Is there a way to fix that? In the awk solution the way to preserve that whitespace is to use sub(/^ */, "&" result, $i) - $i = result $i won't work, because Awk rebuilds the line and thereby discards leading whitespace.

Wintermute Over a year ago

@mklement0 In the sed solution, just remove the \s or [[:space:]], as I wrote there. The $i = result $i does, in fact, work because the leading whitespaces are not removed in field splitting if the field separator is a newline. OP's example output suggests that the whitespaces should be removed, though.

mklement0 Over a year ago

Sorry, missed the hint re removing \s* / [[:space:]]*. However, that doesn't actually work, because it preserves the whitespace after placing <Pass>/ <Fail> at the very beginning of the line. Similarly, re Awk: Both our statements are correct in isolation (thanks for pointing out that -F'\n' does not trim leading/trailing whitespace), but neither applies to the situtation at hand, given that you rebuild the line with result as the first token, which invariably replaces any leading whitespace. Thus, sub(/^ */, "&" result, $i) is the right solution here.

mklement0 Over a year ago

Re multi-character RS values: GNU Awk and Mawk support them, BSD Awk (as also used on OSX) does not.

Wintermute Over a year ago

Oh, interesting. Yes, it works with FreeBSD 9.1, and today I learned that it's not just the branch instructions that need to be split off this way but also the labels. I tried sed -e ':a; $!{ N; ba' -e '}' before and got complaints about an unexpected }.

|

mklement0 · Accepted Answer · 2015-04-06 20:46:26Z

Try the following awk command, which should be POSIX-compliant and preserves leading whitespace:

awk '
 / Physical link is / { ++count }
 /, MTU: / {
    tag = (blockLines[1] ~ /Up$/ && $0 ~ /, MTU: 9000,/ ? "<Pass>" : "<Fail>")
    sub(/^/, "&" tag, blockLines[1])
    sub(/^ +/, "&" tag)
    for (i = 1; i < count; ++i) print blockLines[i]
    count = 0
 }
 count > 0 { blockLines[count++] = $0; next }
 { print }
' file

The basic idea is:

Collect a block of lines - all lines between and including the two lines that must be tagged - in an array without printing them yet.
On reaching the end of the block, determine what tag (fail or pass) must be used
Print all lines in the block, with the first and last one tagged accordingly.

Annotated version of the Awk script only:

 / Physical link is / { ++count } # Start of block
 /, MTU: / {                      # End of block - fail/pass can now be determined
    # Determine whether to apply a fail or a pass tag based on the
    # first and last line in the block.
    tag = (blockLines[1] ~ /Up$/ && $0 ~ /, MTU: 9000,/ ? "<Pass>" : "<Fail>")
    # Prepend tag to 1st line in block
    sub(/^/, "&" tag, blockLines[1])
    # Prepend tag to last line in block, preserving leading whitespace.
    sub(/^ +/, "&" tag)
    # Print all lines in block (except for last one).
    for (i = 1; i < count; ++i) print blockLines[i]
    # Reset block line counter.
    count = 0
 }
 # Inside block: collect lines, do not print yet.
 count > 0 { blockLines[count++] = $0; next }
 # Print last line in block and lines outside of blocks.
 { print }

Collectives™ on Stack Overflow

How to match & then replace multiple lines in shell script

2 Answers 2

10 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

10 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related