python RuntimeError: dictionary changed size during iteration

Question

I have obj like this

{hello: 'world', "foo.0.bar": v1, "foo.0.name": v2, "foo.1.bar": v3}

It should be expand to

{ hello: 'world', foo: [{'bar': v1, 'name': v2}, {bar: v3}]}

I wrote code below, splite by '.', remove old key, append new key if contains '.', but it said RuntimeError: dictionary changed size during iteration

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

def expandField(obj, f, v):
    parts = f.split('.')
    if(len(parts) == 1):
        return
    del obj[f]
    for i in xrange(0, len(parts) - 1):
        f = parts[i]
        currobj = obj.get(f)
        if (currobj == None):
            nextf = parts[i + 1]
            currobj = obj[f] = re.match(r'\d+', nextf) and [] or {}
        obj = currobj
    obj[len(parts) - 1] = v

for k, v in obj.iteritems():

RuntimeError: dictionary changed size during iteration

possible duplicate of How to fix this python error? RuntimeError: dictionary changed size during iteration — Felix Kling
– Felix Kling, Commented Apr 11, 2012 at 14:16
@D.Shawley Ironically this question is now the first hit on Google when searching for the error message. — kasperd
– kasperd, Commented Nov 21, 2014 at 3:23

Scott Hunter · Accepted Answer · 2012-04-11 14:15:36Z

30

Like the message says: you changed the number of entries in obj inside of expandField() while in the middle of looping over this entries in expand.

You might try instead creating a new dictionary of the form you wish, or somehow recording the changes you want to make, and then making them AFTER the loop is done.

answered Apr 11, 2012 at 14:15

Scott Hunter

50k12 gold badges64 silver badges107 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Niklas B. Over a year ago

Yep, a new dictionary is the way to go. This also simplifies the recursion.

Damien · Accepted Answer · 2022-09-12 18:14:53Z

16

You might want to copy your keys in a list and iterate over your dict using the latter, eg:

def expand(obj):
    keys = list(obj.keys())  # freeze keys iterator into a list
    for k in keys:
        expandField(obj, k, v)

I let you analyse if the resulting behavior suits your expected results.

Edited as per comments, thank you !

edited Sep 12, 2022 at 18:14

answered Feb 11, 2014 at 22:07

Damien

1,7342 gold badges20 silver badges27 bronze badges

3 Comments

Kai Carver Over a year ago

might work better withkeys = list(obj.keys()) which copies the keys into a list

s3dev Over a year ago

keys = obj.keys() works nicely. No need to add the memory overhead of converting to a list.

Splines Over a year ago

keys = list(obj.keys()) is necessary if you delete any key of the dict while iterating over it (no matter if you iterate over the dict itself or using its keys). This is because obj.keys() returns just a view on the dict and thus reflects all changes done to the dict. Using the list(...) constructor will actually copy the keys to a new, independent list.

Tomasz Bartkowiak · Accepted Answer · 2020-06-14 13:35:12Z

4

For those experiencing

RuntimeError: dictionary changed size during iteration

also make sure you're not iterating through a defaultdict when trying to access a non-existent key! I caught myself doing that inside the for loop, which caused the defaultdict to create a default value for this key, causing the aforementioned error.

The solution is to convert your defaultdict to dict before looping through it, i.e.

d = defaultdict(int)
d_new = dict(d)

or make sure you're not adding/removing any keys while iterating through it.

answered Jun 14, 2020 at 13:35

Tomasz Bartkowiak

15.4k6 gold badges63 silver badges66 bronze badges

2 Comments

Ashish Ranjan Over a year ago

been scratching my head over this!

judovana Over a year ago

Ty. this was extremly helpfull, Was searching on wrong palces all the tim.

xpros · Accepted Answer · 2013-09-14 21:24:47Z

3

I had a similar issue with wanting to change the dictionary's structure (remove/add) dicts within other dicts.

For my situation I created a deepcopy of the dict. With a deepcopy of my dict, I was able to iterate through and remove keys as needed.Deepcopy - PythonDoc

A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

Hope this helps!

answered Sep 14, 2013 at 21:24

xpros

2,51620 silver badges15 bronze badges

Comments

mausamsion · Accepted Answer · 2020-05-07 02:19:16Z

0

Rewriting this part

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

to the following

def expand(obj):
    keys = obj.keys()
    for k in keys:
        if k in obj:
            expandField(obj, k, v)

shall make it work.

answered May 7, 2020 at 2:19

mausamsion

2301 silver badge8 bronze badges

Collectives™ on Stack Overflow

python RuntimeError: dictionary changed size during iteration

5 Answers 5

1 Comment

3 Comments

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

3 Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related