Tokenize string in python using regular expression

Question

I've input data in the following format, not decided by me

key1: value1 key2: value2 key3: value3 key4 { key11: val11 key22: value22 } key5: value5 ............

The input string will have key values separated by colon or a brace bracket.

I want to tokenize it and I have the following idea: First to have a regular expression parsing data till I find a : or { with priority to { over :

Then split and read till the white space pattern that I said is reached and recursively traverse the whole string

I want to know if I can write a regex like (some_string)(special character pattern) (special character pattern can be : or { with precedence to {)(rest of the string)

If it is a : then for rest of the string, get the string part from ' value1 ' and capture it. Work on the remaining string

If it is a { then traverse till you find } and internally work with : logic defined above.

For eg

a: 1 b: 2 c { d: 3 e: 4 } f: 5

This should give

a:1
b:2
c { d: 3 e: 4 }
f: 5

Is regex a requirement, or would you be OK with a function?

SethMMorton
– SethMMorton

2013-05-31 22:10:54 +00:00
Commented May 31, 2013 at 22:10 — SethMMorton
– SethMMorton, Commented May 31, 2013 at 22:10
@SethMMorton FUnction is also ok

gizgok
– gizgok

2013-05-31 22:12:45 +00:00
Commented May 31, 2013 at 22:12 — gizgok
– gizgok, Commented May 31, 2013 at 22:12

Casimir et Hippolyte · Accepted Answer · 2013-05-31 22:20:02Z

4

You can use this pattern:

[^ ]+(?:: [^ ]+| \{[^}]+\})

example:

import re
test = "a: 1 b: 2 c { d: 3 e: 4 } f: 5"
pattern = re.compile(r"[^ ]+(?:: [^ ]+| \{[^}]+\})")
for match in pattern.findall(test):
    print match

edited May 31, 2013 at 22:20

answered May 31, 2013 at 22:13

Casimir et Hippolyte

90k5 gold badges102 silver badges131 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

gizgok Over a year ago

This only gave the first key and not the remaining keys,

Casimir et Hippolyte Over a year ago

@gizgok: test the example, you must use findall

Markku K. Over a year ago

I downvoted initially because there was no explanation, just the pattern. With the addition of the example, I have reversed my downvote, and in fact upvoted the answer.

Collectives™ on Stack Overflow

Tokenize string in python using regular expression

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related