Python regex search for hexadecimal bytes

Question

I'm trying to search a binary file for a series of hexadecimal values, however, I've run into a few issues that I can't quite solve. (1) I'm not sure how to search the entire file and return all the matches. Currently I have f.seek going only as far as I think the value might be, which is no good. (2) I'd like to return the offset in either decimal or hex where there might be a match, although I get 0 each time, so I'm not sure what I did wrong.

example.bin

AA BB CC DD EE FF AB AC AD AE AF BA BB BC BD BE
BF CA CB CC CD CE CF DA DB DC DD DE DF EA EB EC

code:

# coding: utf-8
import struct
import re

with open("example.bin", "rb") as f:
    f.seek(30)
    num, = struct.unpack(">H", f.read(2))
hexaPattern = re.compile(r'(0xebec)?')
m = re.search(hexaPattern, hex(num))
if m:
   print "found a match:", m.group(1)
   print " match offset:", m.start()

Maybe there's a better way to do all this?

how big is the file?

Urban48
– Urban48

2014-12-29 23:04:13 +00:00
Commented Dec 29, 2014 at 23:04 — Urban48
– Urban48, Commented Dec 29, 2014 at 23:04
File could range in size from 100 KB to 10 MB.

DIF
– DIF

2014-12-29 23:05:42 +00:00
Commented Dec 29, 2014 at 23:05 — DIF
– DIF, Commented Dec 29, 2014 at 23:05

Mark Tolonen · Accepted Answer · 2014-12-30 07:57:42Z

3

I'm not sure how to search the entire file and return all the matches.

I'd like to return the offset in either decimal or hex

import re

f = open('data.txt', 'wb')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.write('\xAA\xBB\xEB\xEC')
f.close()

f = open('data.txt', 'rb')
data = f.read()
f.close()

pattern = "\xEB\xEC"
regex = re.compile(pattern)

for match_obj in regex.finditer(data):
    offset = match_obj.start()
    print "decimal: {}".format(offset)
    print "hex(): " + hex(offset)
    print 'formatted hex: {:02X} \n'.format(offset)

--output:--
decimal: 2
hex(): 0x2
formatted hex: 02 

decimal: 6
hex(): 0x6
formatted hex: 06 

decimal: 10
hex(): 0xa
formatted hex: 0A 

decimal: 14
hex(): 0xe
formatted hex: 0E 

decimal: 18
hex(): 0x12
formatted hex: 12 

decimal: 22
hex(): 0x16
formatted hex: 16 

decimal: 26
hex(): 0x1a
formatted hex: 1A

The positions in the file use 0 based indexing like a list.

e.finditer(pattern, string, flags=0)
Return an iterator yielding MatchObject instances over all non-overlapping matches for the RE pattern in string. The string is scanned left-to-right, and matches are returned in the order found.

Match objects support the following methods and attributes:
start([group])
end([group])
Return the indices of the start and end of the substring matched by group; group defaults to zero (meaning the whole matched substring).

https://docs.python.org/2/library/re.html

edited Dec 30, 2014 at 7:57

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

answered Dec 30, 2014 at 0:37

7stud

48.8k14 gold badges107 silver badges136 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

DIF Over a year ago

nice, that works out quite well. thanks for the explanation, very helpful!

7stud Over a year ago

@DIF, If you don't want the '0x' in front of the offset hex string, you can use format(): print '{:02X}'.format(offset) (also creates two digits for all hex codes).

Urban48 · Accepted Answer · 2014-12-29 23:32:48Z

1

try

import re

with open("example.bin", "rb") as f:
    f1 = re.search(b'\xEB\xEC', f.read())

print "found a match:", f1 .group()
print " match offset:", f1 .start()

edited Dec 29, 2014 at 23:32

answered Dec 29, 2014 at 23:24

Urban48

1,4861 gold badge15 silver badges28 bronze badges

2 Comments

DIF Over a year ago

thanks, that's almost perfect. is there a way to make the f1.group() display as hexadecimal?

qff Over a year ago

Print f1.group() as hexadecimal: print(f1.group().hex()).

Collectives™ on Stack Overflow

Python regex search for hexadecimal bytes

2 Answers 2

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related