Python with regex to replace strings recursively

Question

I've tried different ideas for this but haven't been successful yet. Hence this post.

Some background: I am trying to decipher a Makefile that might include variables from a separate file. I've managed to read all the variables in Makefile and also its includes successfully into a python dictionary. But now I find that each of the values essentially reference other variables that are part of the dictionary. What I want to do is to unroll all the values in the dictionary to have text that are independent of other key/value pairs. This definitely involves recursion (IMHO) but I am very interested in hearing any other suggestions.

Note that not all variables might have a value associated with it. In this case, replace the key with a NULL string.

Now for some code to demonstrate what is said above:

Let a list of key,value pair be

*A = -L${F} ${B} ${D},

*B = -L/myhome,

*F = /usr/lib

I want to write a python script (possibly with regex) to recursively replace the values matching '${XXX}' with the corresponding key until there are no more values available that match the prescribed pattern (i.e., everything is unrolled). Since D doesn't have a value associated with it, I want the value of A to eventually be (for example)

*A = -L/usr/lib -L/myhome

Thanks in advance. Any help would be much appreciated.

hamstergene · Accepted Answer · 2011-08-17 04:50:38Z

4

Take advantage of re.subn, which returns number of replacements (so you know when to stop) and accepts function for repl argument (to select value from vars dictionary):

import re

vs = { 
    'A' : '-L${F} ${B} ${D}',
    'B' : '-L/myhome',
    'F' : '/usr/lib',
}

while 1:
    treps = 0
    for k in vs:
        ns, nreps = re.subn(r'''\${(\w+)}''', lambda match: vs.get(match.group(1), ''), vs[k])
        if nreps: vs[k] = ns
        treps += nreps
    if not treps: break

print(vs)
# {'A': '-L/usr/lib -L/myhome ', 'B': '-L/myhome', 'F': '/usr/lib'}

Take care that the program above will never end if A=${A}, or if A=${B} and B=${A}. You did not specify what should happen in such cases.

answered Aug 17, 2011 at 4:50

hamstergene

24.5k5 gold badges60 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

VSM Over a year ago

This works perfectly. Even for the degenerate case when an environment variable is not defined :) I understand your cyclic reference but hopefully, in the use cases I have, this will never happen.

bd808 · Accepted Answer · 2011-08-17 05:23:36Z

2

With a helper function to do the recursive expansion you can use re.sub to replace all non-overlapping matches in each value:

import re
RE_VAL = re.compile(r'\${(.*?)}')

def expand (val, src):
  return RE_VAL.sub(lambda m: expand(src.get(m.group(1), ''), src), val)

def main ():
  vals = {
      'A': '-L${F} ${B} ${D}',
      'B': '-L/myhome',
      'D': '${E}',
      'E': '${G}',
      'F': '/usr/lib',
      'G': '-O',
  }

  for k,v in vals.iteritems():
    vals[k] = expand(v, vals)
  print vals
  # {'A': '-L/usr/lib -L/myhome -O', 'B': '-L/myhome', 'E': '-O', 'D': '-O', 'G': '-O', 'F': '/usr/lib'}

answered Aug 17, 2011 at 5:23

bd808

1,8432 gold badges24 silver badges32 bronze badges

2 Comments

VSM Over a year ago

Very nice. This is lot more elegant than what I conjured. And produces what I need. Just an additional question here: If I had an additional dictionary that served as reference, then can you actually check if a key exists in the reference first before substituting a null string for the key (which is what happens now).

bd808 Over a year ago

Sure, just pass your "defaults" dict as an arg to expand() and replace the empty string in the src.get() call with an empty safe call to the second dict. You end up with: src.get(m.group(1), defaults.get(m.group(1), ''))

Charles Ma · Accepted Answer · 2011-08-17 04:50:18Z

0

Something like this:

def unroll(stuff):

   # Code to unroll.
   # if something had been replaced:
      replaced = True
   # else
      replaced = False

   return stuff, replaced


def main():

   stuff, replaced = unroll(stuff)
   while replaced:
      stuff, replaced = unroll(stuff)

Be careful of infinite replacement loops though!

answered Aug 17, 2011 at 4:50

Charles Ma

49.6k22 gold badges92 silver badges102 bronze badges

Collectives™ on Stack Overflow

Python with regex to replace strings recursively

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related