326

I am passing a list of regex patterns to grep to check against a syslog file. They are usually matching an IP address and log entry;

grep "1\.2\.3\.4.*Has exploded" syslog.log

It's just a list of patterns like the "1\.2\.3\.4.*Has exploded" part I am passing, in a loop, so I can't pass "-v", for example.

I am confused trying to do the inverse of the above, and not match lines with a certain IP address and error so "!1.2.3.4.*Has exploded" will match syslog lines for anything other than 1.2.3.4 telling me it has exploded. I must be able to include an IP address to not match.

I have seen various similar posts on Stack Overflow. However, they use regex patterns that I can't seem to get to work with grep. What would be a working example for grep?

This is happening in a script like this;

patterns[1]="1\.2\.3\.4.*Has exploded"
patterns[2]="5\.6\.7\.8.*Has died"
patterns[3]="\!9\.10\.11\.12.*Has exploded"

for i in {1..3}
do
  grep "${patterns[$i]}" logfile.log
done
Improve this question
4
  • Do you mean you sometimes want to match a pattern, but other times want to match everything except a certain pattern? (this seems like an odd requirement, but whatever). In that case, why don't you iterate over two different lists of patterns? Commented May 2, 2012 at 10:42
  • Well I'm not very knowledgeable about regex; I don't want to grep for "Has Exploded" because I don't want to know this about every logging device, so can I somehow grep for "Has Exploded" and !9.10.11.12 in one statement? Commented May 2, 2012 at 12:06
  • If you absolutely must do it in one statement, negative lookbehinds are the way to go, as Neil suggests. See my comment there. Commented May 2, 2012 at 12:57
  • Use PCRE-style regex matching, and a negative lookahead assertion, as per @Neil 's answer: patterns[3]="\!9\.10\.11\.12.*Has exploded" changes to patterns[3]="(?<!9\.10\.11\.12).*Has exploded" and grep "${patterns[$i]}" logfile.log changes to grep -P "${patterns[$i]}" logfile.log PCRE assumes more metacharacters by default, so some of the escapes may need to be removed from other matching expressions. Commented Jun 5, 2019 at 16:45

4 Answers 4

Reset to default
597

grep matches, grep -v does the inverse. If you need to "match A but not B" you usually use pipes:

grep "${PATT}" file | grep -v "${NOTPATT}"
Improve this answer
Sign up to request clarification or add additional context in comments.

10 Comments

This is going into the middle of a loop as I mentioned and I'm just passing the PATTERN to grep so I can't use "-v" as I mentioned. I'm just looping round a list of PATTERNs and passing to grep.
You can indeed use -v and you can use it in a loop. Perhaps you need to be more specific about your limitations, or perhaps you have a misconception about how your script should work. Try posting some code.
Thanks beerbajay, I have added a code snipped to the original post to give some context. Do you see what I mean now?
But what if A is composed of B? In other words, what if I want to match lines with no A and lines with AB ? A pipe will not work.
grep -v for me DOES NOT work: it still outputs the lines containing the words in the NOTPATT
|
21 
(?<!1\.2\.3\.4).*Has exploded

You need to run this with -P to have negative lookbehind (Perl regular expression), so the command is:

grep -P '(?<!1\.2\.3\.4).*Has exploded' test.log

Try this. It uses negative lookbehind to ignore the line if it is preceded by 1.2.3.4.

Improve this answer

5 Comments

I'm pretty sure that grep doesn't support lookaround. Unless you're using Gnu grep and use the --P parameter to make it use a PCRE engine.
No, grep doesn't support this type of Regex; $grep -P (?<\!1\.2\.3\.4) test.log -bash: syntax error near unexpected token `('
You're going to need to quote the regex if it contains characters which would be interpreted by the shell.
correct quoting: grep -P '(?<!1\.2\.3\.4) Has exploded' test.log Note that the lookbehind only works on the characters immediately preceding the matching part of the expression, so if there's other things between the address and message, e.g. 1.2.3.4 FOO Has exploded, this won't work.
@TimPietzcker, very observant. I'll add that to the question. Also, please note that there is a .* after the negative lookbehind since his example also has it, I imagine there might be other text in between.
20

It seems no one has posted a blend of the best of all answers, regex (-E) with match inversion (-v)

grep -Ev 'pattern1|pattern2|pattern3' file

Notably, no lookarounds required, so this works if your grep version doesn't have -P available.

Improve this answer

Comments

2 
patterns[1]="1\.2\.3\.4.*Has exploded"
patterns[2]="5\.6\.7\.8.*Has died"
patterns[3]="\!9\.10\.11\.12.*Has exploded"

for i in {1..3}
  do
grep "${patterns[$i]}" logfile.log
done

should be the the same as

egrep "(1\.2\.3\.4.*Has exploded|5\.6\.7\.8.*Has died)" logfile.log | egrep -v "9\.10\.11\.12.*Has exploded"
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.