python regex in pyparsing

Question

How do you make the below regex be used in pyparsing? It should return a list of tokens given the regex.

Any help would be greatly appreciated! Thank you!

python regex example in the shell:

>>> re.split("(\w+)(lab)(\d+)", "abclab1", 3)
>>> ['', 'abc', 'lab', '1', '']

I tried this in pyparsing, but I can't seem to figure out how to get it right because the first match is being greedy, i.e the first token will be 'abclab' instead of two tokens 'abc' and 'lab'.

pyparsing example (high level, i.e non working code):

name = 'abclab1'
location = Word(alphas).setResultsName('location')
lab = CaselessLiteral('lab').setResultsName('environment')
identifier = Word(nums).setResultsName('identifier')
expr = location + lab + identifier
match, start, end = expr.scanString(name).next()
print match.asDict()

PaulMcG · Accepted Answer · 2012-06-20 18:59:01Z

Pyparsing's classes are pretty much left-to-right, with lookahead implemented using explicit expressions like FollowedBy (for positive lookahead) and NotAny or the '~' operator (for negative lookahead). This allows you to detect a terminator which would normally match an item that is being repeated. For instance, OneOrMore(Word(alphas)) + Literal('end') will never find a match in strings like "start blah blah end", because the terminating 'end' will get swallowed up in the repetition expression in OneOrMore. The fix is to add negative lookahead in the expression being repeated: OneOrMore(~Literal('end') + Word(alphas)) + Literal('end') - that is, before reading another word composed of alphas, first make sure it is not the word 'end'.

This breaks down when the repetition is within a pyparsing class, like Word. Word(alphas) will continue to read alpha characters as long as there is no whitespace to stop the word. You would have to break into this repetition using something very expensive, like Combine(OneOrMore(~Literal('lab') + Word(alphas, exact=1))) - I say expensive because composition of simple tokens using complex Combine expressions will make for a slow parser.

You might be able to compromise by using a regex wrapped in a pyparsing Regex object:

>>> labword = Regex(r'(\w+)(lab)(\d+)')
>>> print labword.parseString("abclab1").dump()
['abclab1']

This does the right kind of grouping and detection, but does not expose the groups themselves. To do that, add names to each group - pyparsing will treat these like results names, and give you access to the individual fields, just as if you had called setResultsName:

>>> labword = Regex(r'(?P<locn>\w+)(?P<env>lab)(?P<identifier>\d+)')
>>> print labword.parseString("abclab1").dump()
['abclab1']
- env: lab
- identifier: 1
- locn: abc
>>> print labword.parseString("abclab1").asDict()
{'identifier': '1', 'locn': 'abc', 'env': 'lab'}

The only other non-regex approach I can think of would be to define an expression to read the whole string, and then break up the parts in a parse action.

nrek · Accepted Answer · 2012-06-20 09:30:04Z

0

If you strip the subgroup sign(the parenthesis), you'll get the right answer:)

>>> re.split("\w+lab\d+", "abclab1")
['', '']

answered Jun 20, 2012 at 9:30

nrek

2713 silver badges12 bronze badges

1 Comment

user1468650 Over a year ago

thanks for the answer, but what i am trying to do is use pyparsing to do pretty much do what the regex does. so using pyparsing, it should return a dict like ... { location: abc, enviornment: lab, identifier: 1 }

Collectives™ on Stack Overflow

python regex in pyparsing

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related