matching list of substrings to a list of strings in Python

Question

What is a good pythonic way to match a list of substrings to a list of strings, like the following:

if 'sub1' in str1 or 'sub2' in str1 or ... 'subN' in str1 or\
   'sub1' in str2 or 'sub2' in str2 or ... 'subN' in str2 or\
   ...
   'sub1' in strM or 'sub2' in strM or ... 'subN' in strM:

One way is to unite them with list comprehension, like this:

strList = [str1, str2, ..., strM]
subList = ['sub1', ..., 'subN']
if any(sub in str for sub in subList for str in strList):

Is there anything better, like a library function perhaps, to absorb one of the dimensions?

Thank you very much.

If the substrings are short enough (and not too many), you could compose them into a regular expression: sub1|sub2|sub3|...|subN — alexis
– alexis, Commented Jun 21, 2013 at 12:14
Depending on the number and size of strings, it might be faster to concatenate all str[1..n]s into one large string and then check for substring matches once, using regex or any(... in ...). — Tim Pietzcker
– Tim Pietzcker, Commented Jun 21, 2013 at 12:30
@TimPietzcker that's an interesting thought. But the advantage for my specific case is that substrings are static, while strings are dynamic, so I can precompute the substring RE once and use it all over the place. But this is still quite original, thank you very much. — gt6989b
– gt6989b, Commented Jun 21, 2013 at 12:38

alexis · Accepted Answer · 2013-06-21 12:31:32Z

5

You could compile the substrings into a regular expression, and use that to search each string. If you don't have so many substrings that the RE exceeds internal limits, this is probably the most efficient way.

pattern = "|".join(re.escape(s) for s in subList)
crexp = re.compile(pattern)
if any(crexp.search(s) for s in strList):
    ...

edited Jun 21, 2013 at 12:31

answered Jun 21, 2013 at 12:22

alexis

50.4k18 gold badges108 silver badges173 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2017-05-23 12:29:13Z

3

As described in this answer, a regular expression would be the way to go, since these are modeled as a DFA that can check for all substrings at the same time. You should probably read that answer, as it is quite in-depth.

edited May 23, 2017 at 12:29

CommunityBot

11 silver badge

answered Jun 21, 2013 at 12:21

Imre Kerr

2,41316 silver badges35 bronze badges

1 Comment

Tim Pietzcker Over a year ago

Python's regex engine doesn't implement a DFA engine, but I support your conclusion.

Collectives™ on Stack Overflow

matching list of substrings to a list of strings in Python

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related