If I have a file example apache log file
How to extract the top most frequent error messages in a unix log file with no timestamps
the key is most frequent error message should come on top of the list
cat /tmp/file:
ERROR 1 1234
ERROR 2 1234
ERROR 3 1234
ERROR 4 1234
ERROR 4 1234
ERROR 3 1234
ERROR 2 1234
ERROR 5 1234
ERROR 1 1234
ERROR 4 1234
ERROR 1 1234
ERROR 1 1234
ERROR 1 1234
ERROR 3 1234
ERROR 2 1234
ERROR 1 1234
ERROR 4 1234
ERROR 1 1234
ERROR 4 1234
ERROR 1 1234
ERROR 2 1234
grep "ERROR" /tmp/file | sort | uniq -c | sort -r:
8 ERROR 1 1234
5 ERROR 4 1234
4 ERROR 2 1234
3 ERROR 3 1234
1 ERROR 5 1234
first column shows how many occurrences of each string were found Explanation:
grep "ERROR" /tmp/file\ # select only ERROR string
| sort\ # order
| uniq -c\ # count duplicate items
| sort -rn # reverse order and use numeric sort
for top 5 errors, you can add |head -n5
I think you have to chop off timestamp to get unique error
grep 'error message' /logfiles | cut -d' ' -f6- | sort | uniq -c | sort -nr
You might be interested in "sorting" all you system.log errors by a specific date + time too?
try: (for date only search)
grep -i "Jul 18" /var/log/*.log | sort | uniq -c | sort -n
try: (for for a specific date + time search)
grep -i "Jul 18 16:" /var/log/*.log | sort | uniq -c | sort -n
NOTE: for a specific date first do a cat *.log and then look for what the header says it might be different depending on the O/S. The above example is for UNIX/MacOS and you have to "manually" edit the "Jul 18 16:" with what it says in your "cat /var/log/system.log" header for your particular O/S.
Hope this helps!
:)
grep message logfile | sort | uniq -c | sort -n | headheadshould readtail.grep -vi -e 'info' -e 'warn'