Find multi-line text & replace it, using regex, in shell script

Question

I am trying to find a pattern of two consecutive lines, where the first line is a fixed string and the second has a part substring I like to replace.

This is to be done in sh or bash on macOS.

If I had a regex tool at hand that would operate on the entire text, this would be easy for me. However, all I find is bash's simple text replacement - which doesn't work with regex, and sed, which is line oriented.

I suspect that I can use sed in a way where it first finds a matching first line, and only then looks to replace the following line if its pattern also matches, but I cannot figure this out.

Or are there other tools present on macOS that would let me do a regex-based search-and-replace over an entire file or a string? Maybe with Python (v2.7 and v3 is installed)?

Here's a sample text and how I like it modified:

  keyA
  value:474
  keyB
  value:474    <-- only this shall be replaced (follows "keyB")
  keyC
  value:474
  keyB
  value:474

Now, I want to find all occurances where the first line is "keyB" and the following one is "value:474", and then replace that second line with another value, e.g. "value:888".

As a regex that ignores line separators, I'd write this:

Search: (\bkeyB\n\s*value):474
Replace: $1:888

So, basically, I find the pattern before the 474, and then replace it with the same pattern plus the new number 888, thereby preserving the original indentation (which is variable).

Try sed '/keyB$/{n;s/$.*$:[0-9]*/\1:888/}' file or sed -e '/keyB$/{n' -e 's/$.*$:[0-9]*/\1:888/' -e '}' file — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Feb 3, 2021 at 22:27

Wiktor Stribiżew · Accepted Answer · 2021-02-03 23:20:17Z

5

You can use

sed -e '/keyB$/{n' -e 's/\(.*\):[0-9]*/\1:888/' -e '}' file
# Or, to replace the contents of the file inline in FreeBSD sed:
sed -i '' -e '/keyB$/{n' -e 's/\(.*\):[0-9]*/\1:888/' -e '}' file

Details:

/keyB$/ - finds all lines that end with keyB
n - empties the current pattern space and reads the next line into it
s/$.*$:[0-9]*/\1:888/ - find any text up to the last : + zero or more digits capturing that text into Group 1, and replaces with the contents of the group and :888.

The {...} create a block that is executed only once the /keyB$/ condition is met.

See an online sed demo.

answered Feb 3, 2021 at 23:20

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Thomas Tempelmann Over a year ago

While the Perl solution is more directly giving me what I wanted to do based on my own skills, I appreciate learning a bit more about sed and its processing of consecutive lines.

Peter Thoeny · Accepted Answer · 2021-02-03 22:48:28Z

2

Use a perl one-liner with -0777 to scan over multiple lines:

$ # inline edit:
$ perl -0777 -i -pe 's/\bkeyB\s*value):\d*/$1:888/' file.txt
$ # to stdout:
$ cat file.txt | perl -0777 -pe 's/\bkeyB\s*value):\d*/$1:888/'

answered Feb 3, 2021 at 22:48

Peter Thoeny

7,6381 gold badge16 silver badges24 bronze badges

2 Comments

Thomas Tempelmann Over a year ago

Ahh, Perl. My long-avoided nemesis :) What's the "-0777" for? Not permissions, is it?

Peter Thoeny Over a year ago

-0777 changes the line separator to undef, to feed all the lines to perl in one go.

M. Nejat Aydin · Accepted Answer · 2021-02-04 01:11:06Z

1

In plain bash:

#!/bin/bash

keypattern='^[[:blank:]]*keyB$'
valpattern='(.*):'
replacement=888

while read -r; do
    printf '%s\n' "$REPLY"
    if [[ $REPLY =~ $keypattern ]]; then
        read -r
        if [[ $REPLY =~ $valpattern ]]; then
            printf '%s%s\n' "${BASH_REMATCH[0]}" "$replacement"
        else
            printf '%s\n' "$REPLY"
        fi
    fi
done < file

answered Feb 4, 2021 at 1:11

M. Nejat Aydin

10.3k1 gold badge10 silver badges22 bronze badges

Collectives™ on Stack Overflow

Find multi-line text & replace it, using regex, in shell script

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related