I have a file that contains:
something
something else
something else again
I need a bash command, sed/grep w.e that will produce the following output
something
something else
something else again
In other words, I need to replace multiple blank lines with just a single blank line. grep/sed are line based. I've never found a BASH solution that would work on multi-line regex patterns.
For BSD-derived systems (including GNU):
You just need cat with the -s option which causes it to remove repeated empty lines from its output:
cat -s
From man page: -s --squeeze-blank: suppress repeated empty output lines.
cat -s file1 > file2
cat -s worked fine for me.
cat on Solaris, the -s option has a different meaning "-s cat is silent about non-existent files"
some_output|cat -s.I just solved this problem by sed. Even if this is a 7 year old question, someone may find this helpful, so I am writing my solution by sed here:
sed 'N;/^\n$/D;P;D;'
sed '$!N;/^\n$/{$q;D;};P;D;' prevents the last line from being deleted on OSX sed. Also works on gnu sed.
grep -A1 . <yourfile> | grep -v "^--$"
This grep solution works assuming you want the following:
Input
line1
line2
line3
line4
line5Output
line1
line2
line3
line4
line5
Actually, if you replace multiple newlines with a single newline, the output would be:
something
something else
something else again
You can achieve this by:
sed /^$/d FILE
Use awk:
awk '{ /^\s*$/?b++:b=0; if (b<=1) print }' file
Breakdown:
/^\s*$/?b++:b=0
- ? : the ternary operator
- /^\s*$/ matches a blank line
- b variable that counts consecutive blank lines (b++).
however, if the current line is non-blank, b is reset to 0.
if (b<=1) print
print if the current line is non-blank (b==0)
or if there is only one blank line (b==1).
By adjusting the regex, you can generalize it to other scenarios like squeezing multiple blank lines (">") in email: https://stackoverflow.com/a/59189823/12483961
A solution with awk, which replaces several blank lines with a single blank line:
awk 'BEGIN{bl=0}/^$/{bl++;if(bl==1)print;else next}/^..*$/{bl=0;print}' myfile
Usually, if I find that sed can't do something I need, I turn to awk:
awk '
BEGIN {
blank = 0;
}
/^[[:blank:]]*$/ {
if (!blank) {
print;
}
blank = 1;
next;
}
{
print;
blank = 0;
}' file
If someone want use perl
perl -00pe0 < file
will do the same, as cat -s :)
This uses marco's solution on multiple files:
for i in *; do FILE=$(cat -s "$i"); echo "$FILE" > "$i"; done
Use python:
s = file("filename.txt").read()
while "\n\n\n" in s: s = s.replace("\n\n\n", "\n\n")
import sys
sys.stdout.write(s)
Python, with regular expression:
import re
import sys
sys.stdout.write(re.sub('\n{2,}','\n\n', sys.stdin.read()))
I take it that you'll probably want to remove lines that only have whitespace.
That can be done with:
sed /^[:space:]*$/d FILE
Pipelining it to |uniq may be solution (if other than empty lines don't duplicate)
