6

I am creating executable which will be executed with /bin/sh or /bin/bash script,
I have a file which contains a structure like, there will be only one #start and #end tag in the config file, and I want to replace text in between those tags,

...

#start
FirewallRuleSet global {
    FirewallRule allow tcp to google.com
    FirewallRule allow tcp to facebook.com

#more rules
}
#end

FirewallRuleSet known-users {
    FirewallRule allow to 0.0.0.0/0
}

...

Desired output will be, 

...

#start
FirewallRuleSet global {
    FirewallRule allow tcp to google.com
    FirewallRule deny tcp to facebook.com
    FirewallRule deny tcp to twitter.com
    FirewallRule allow tcp to exaple.com



#more rules
}
#end

FirewallRuleSet known-users {
    FirewallRule allow to 0.0.0.0/0
}

...

How can I replace the whole text between #start and #end with some new text? I just want to add or remove Rules from this config file.

This is part of a config file and I want to modify url allowed inside that texts.

Improve this question
2
  • @don_crissti there are no multiple tags, added exampe output. Commented Mar 25, 2016 at 11:50
  • 1
    OK. I've edited Lucas' answer as it was very close to what you want. Commented Mar 25, 2016 at 11:58

4 Answers 4

Reset to default
6

Use

sed '/#start/,/#end/replace_command'

For example, if the file is called myconfig, and you want to replace "allow" with "deny" in that section, you could say

sed '/#start/,/#end/s/allow/deny/' myconfig

That would leave the file untouched, and display on the standard output what the file would look like after the modification.  You should probably do that first, to verify that you've got the command right.  If you want to actually change the file, add the -i option:

sed -i '/#start/,/#end/s/allow/deny/' myconfig

If you want to replace the whole text (all the text) between those two lines, you can do something slightly simpler than Lucas's answer:

sed '/#start/,/#end/c\
New text line 1\
New text line 2\
        ︙      \
New text line n-1\
New text line n (last)'                                   ← Close quote; no backslash here

c is the change command in sed (and ed); it means "replace entire line(s)".  You cannot simply leave the #start and #end lines untouched.  If you want to keep them, you must re-insert them:

sed -i '/#start/,/#end/c\
#start\
FirewallRuleSet global {\
    FirewallRule allow tcp to google.com\
    FirewallRule deny tcp to facebook.com\
                      ︙                 \
\
#more rules\
}\
#end' myconfig

/#start/,/#end/ specifies a range — the lines from the first line that contains #start through the first line after that that contains #end.  If you need to find lines that contain those strings and nothing else, use /^#start$/,/^#end$/.

Improve this answer
3
  • Thanks @G-man. I tried this once before, but I want to replace whole text between those #start and #end Commented Mar 25, 2016 at 9:28
  • doesnt work for me Commented Dec 31, 2020 at 12:27
  • @NadavB: Well, it has worked for a few people, and I don’t see any other complaints.  “doesn’t work” is very broad.  If you would say more about your situation (what you want to do, with what input), what you tried and what actually happened, I might be able to give you some help. But we discourage such conversations in comments; so they are limited to 600 characters.  You should probably ask a new question (with the information I asked for, and a link back to this question) and then post another comment here to alert me to it. Commented Dec 31, 2020 at 21:17
4

Based on G-Man's answer and the comment:

sed -i '/#start/,/#end/ {
//!d
/#start/a\
some new text\
more lines\
end of new text (no backslash here!)
}' myconfig

Explanation:

  1. /#start/,/#end/ { .... } executes the commands in curly brackets for every line between the text "#start" and "#end" (inclusive). Compare G-Man's answer.
  2. a is the append command. It is only executed on the line matching "#start" in order to add the new text. It appends the lines until a line that is not terminated with a backslash+newline.
Improve this answer
2
  • This answer could be improved if you would edit it to explain how it works in comparison to mine — specifically, (1) the function of the {…}, (2) the function of the //!d, and (3) why it explicitly references the /#start/ pattern twice. Commented Mar 29, 2016 at 19:00
  • 1
    @don_chrissti: how does the //!d part in your edit work? I am wondering why it does not delete the "#start" and "#end" lines. So my question is: What does // match? The empty string or some previous regex (which)? Commented Mar 30, 2016 at 22:43
0

Because you have two states that the presence of two different lines shift between this is a job for awk (or perl or python). Since my current language is python the program would look like this:

import sys

rule_file = sys.argv[1]
new_rules = sys.argv[2]

mode = "save_rules"
for line in open(rule_file):
    line = line.strip()
    if line == "#start":
        print line
        print new_rules
        mode = "replace_rules"
    elif line == "#end":
        mode = "save_rules"
        print line
    elif mode == "save_rules":
        print line

Now save this into rule_replace.py and call it with

python rule_replace.py my_rule_file.txt 'FirewallRuleSet global {
FirewallRule allow tcp to google.com
FirewallRule deny tcp to facebook.com
FirewallRule deny tcp to twitter.com
FirewallRule allow tcp to exaple.com



#more rules
}' >new_rules.txt

Of course you do not have to put the new rules on the command line, I am assuming you would have them in a shell variable and then the call would look like:

$ python rule_replace.py my_rule_file.txt $new_rules

Note that this script, while it works, is not a solution for something in production. It does not catch any errors (for instance if the source file is not there). It assumes without checking that a #start line is always followed by an #end line and that the lines are exactly as you described. It also could use some logging if going into production.

Improve this answer
0

Building off G-Man's answer, here's how to do this AND use custom delimiters AND variable substitution (due to the complexity this wasn't immediately obvious)

Given inputfile.txt:

Line1
Line2
<!-- /START_MARKER/ -->
Line4
Line5
Line6
<!-- /END_MARKER/ -->
Line8
Line9

Use these commands/script:

customtext="Any kind of variable text here"

sed -i '\%<!-- /START_MARKER/ -->%,\%<!-- /END_MARKER/ -->%c\
foo '"$customtext"' abcxyz whatever' inputfile.txt

Results in:

Line1
Line2
Any kind of variable text here
Line8
Line9

There's a lot going on here, explained below:

' - Using single quotes must be done to wrap everything

\% - When using a custom delimiter (e.g. if your marker has / you MUST use a different delimiter) it has to be escaped here

<!-- /START_MARKER/ --> - the literal start marker, in this case it's a whacky example for demonstration purposes.

% - sed delimiter, not escaped

, - comma (?)

\% - sed delimiter again, this time MUST be escaped

<!-- /END_MARKER/ --> - Literal ending marker

%c - sed delimiter (not escaped) and c

\ - backslash, signifying the following text/lines should be used as the replacement

foo - some replacement text

'"$customtext"' - custom variable text, note the use of ' and then " to enable variable replacement, then " and ' to return to free text mode

abcxyz whatever - some additional replacement text

' - closing single quote

inputfile.txt - the file name

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.