Bash matching binary pattern

Question

I want to check inside a file if it matches a binary pattern.

For that, I'm using clamAV signature database

Trojan.Bancos-166:1:*:3d415d736715ab5ee347238cacac61c7123fe35427224d25253c7b035558baf19e54e8d1a82742d6a7b37afc6d91015f751de1102d0a31e66ec33b74034b1ab471cc1381884dfdf0bb3e4233bd075fef235f342302ffd72ecabfa5aedf1b3dc99b3348346db4d9001026aef44c592fee61493f7262ad2bd1bce8a7ce60d81022533f6473ae184935f25cf6cc07c3aebfdf70a5a09139

I code this to retrieve the hex string representation signature

signature=$(echo "$line" |awk -F':' '{ print $4 }')

Moreover I change hex string to binary

printf -v variable $(sed 's/\(..\)/\\x\1/g;' <<< "$signature")

Until here It works perfectly.

Finally I would like to check if my file ( *$raw_file_path* ) matches my binary pattern (now in $variable) I try this

test_var=$(grep -qU "$variable" "$raw_file_path")

or

test_var=$(grep -qU --regexp="$variable" "$raw_file_path")

I don't know why it doesn't work, Grep doesn't match anything . And sometimes some errors:

grep: Trailing backslash
grep: Invalid regular expression

I know it deals with pattern matching problems. In my test I don't want use regular expression.

If you have any idea, or other bash tool. Thanks.

to follow on with @Sampson-chen 's answer, you'll also save yourself pain by confirming what is inside $( ... ) is working correctly before you add the extra step of saving the output to a variable. Good luck. — shellter
– shellter, Commented Nov 20, 2012 at 19:28

sampson-chen · Accepted Answer · 2012-11-20 19:08:58Z

2

You are currently using the --quiet option for grep by specifying q in -qU. This prevents grep from printing anything to stdout, therefore nothing will be saved to test_var.

Change your code to:

test_var=$(grep -UE "$variable" "$raw_file_path")

edited Nov 20, 2012 at 19:08

answered Nov 20, 2012 at 19:01

sampson-chen

47.6k13 gold badges87 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

koola · Accepted Answer · 2012-11-23 03:27:13Z

1

First the extra sub-shell can be avoided:

#!/bin/bash
signature="Trojan.Bancos-166:1:*:3d415d736715ab5ee347238cacac61c7123fe35427224d25253c7b035558baf19e54e8d1a82742d6a7b37afc6d91015f751de1102d0a31e66ec33b74034b1ab471cc1381884dfdf0bb3e4233bd075fef235f342302ffd72ecabfa5aedf1b3dc99b3348346db4d9001026aef44c592fee61493f7262ad2bd1bce8a7ce60d81022533f6473ae184935f25cf6cc07c3aebfdf70a5a09139"
variable=$(echo "${signature//*:/}" | sed 's/\(..\)/\\x\1/g;')

Require only confirmation of a match:

if grep -qU "$variable" "$raw_file_path"; then
    # matches
fi

Or require the result for further processing:

test_var=$(grep -U "$variable" "$raw_file_path")
# contents of match in test_var

When returning to a variable, greps -q opt suppresses stdout

Edit

Tested working example

> signature="Trojan.Bancos-166:1:All_text before-the last : should be trimed:3d415d736715ab5ee347238cacac61c7123fe35427224d25253c7b035558baf19e54e8d1a82742d6a7b37afc6d91015f751de1102d0a31e66ec33b74034b1ab471cc1381884dfdf0bb3e4233bd075fef235f342302ffd72ecabfa5aedf1b3dc99b3348346db4d9001026aef44c592fee61493f7262ad2bd1bce8a7ce60d81022533f6473ae184935f25cf6cc07c3aebfdf70a5a09139" \
> hex_string=$( echo "${signature//*:/}" | sed 's/\(..\)/\\x\1/g;' ) \
> echo "$hex_string"
\x3d\x41\x5d\x73\x67\x15\xab\x5e\xe3\x47\x23\x8c\xac\xac\x61\xc7\x12\x3f\xe3\x54\x27\x22\x4d\x25\x25\x3c\x7b\x03\x55\x58\xba\xf1\x9e\x54\xe8\xd1\xa8\x27\x42\xd6\xa7\xb3\x7a\xfc\x6d\x91\x01\x5f\x75\x1d\xe1\x10\x2d\x0a\x31\xe6\x6e\xc3\x3b\x74\x03\x4b\x1a\xb4\x71\xcc\x13\x81\x88\x4d\xfd\xf0\xbb\x3e\x42\x33\xbd\x07\x5f\xef\x23\x5f\x34\x23\x02\xff\xd7\x2e\xca\xbf\xa5\xae\xdf\x1b\x3d\xc9\x9b\x33\x48\x34\x6d\xb4\xd9\x00\x10\x26\xae\xf4\x4c\x59\x2f\xee\x61\x49\x3f\x72\x62\xad\x2b\xd1\xbc\xe8\xa7\xce\x60\xd8\x10\x22\x53\x3f\x64\x73\xae\x18\x49\x35\xf2\x5c\xf6\xcc\x07\xc3\xae\xbf\xdf\x70\xa5\xa0\x91\x39

edited Nov 23, 2012 at 3:27

answered Nov 21, 2012 at 6:30

koola

1,7641 gold badge13 silver badges15 bronze badges

4 Comments

user1778354 Over a year ago

I don't have always * followed by :

koola Over a year ago

As long as you have : before the signature, it will work given // dictates longest match followed by any character * and : Everything before the last : and including : is removed.

user1778354 Over a year ago

it s about grep fonction. No pattern detected or all files detected.

koola Over a year ago

Try add -o --binary-files=binary or alternatively specify -a which process a binary file as if it were text.

Collectives™ on Stack Overflow

Bash matching binary pattern

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related