python: add one element to an existing dictionary from list loop

Question

I need to export data from one database into another database. The dataobject needs to be mapped to a dictionary.

each row needs to be saved twice with different values from a list foobar=['foo','bar'].

foobar=['foo','bar']
data = []
for q in queryset:
    row = {"id": q.id,
           "created_at": q.created_at}
    for f in foobar:
        row['index'] = f
        data.append(row)
bulksave(data)

this doesn't give the desired result:

 print data
 [{'id': 1, 'created_at': '2017-01-01', 'index': 'bar'},
  {'id': 2, 'created_at': '2017-01-02', 'index': 'bar'}]

Where the desired output would have 'foo' and 'bar' as the index. How do I get this to work? Taking a different approach would be to loop differently:

for f in foobar:
    for q in queryset

this works, but takes twice the time because each element in the queryset will be evaluated twice.

You mean you want: [{'id': 1, 'created_at': '2017-01-01', 'index': 'foo'}, {'id': 2, 'created_at': '2017-01-02', 'index': 'bar'}]? — dgg32
– dgg32, Commented Mar 25, 2017 at 17:58
Are you sure about that print data? It should have 4 items, but confusingly its only two unique items with each repeated in the list. — tdelaney
– tdelaney, Commented Mar 25, 2017 at 18:03
@dgg32 - I'm not sure about that either. If queryset has two entries, data should have 4 unique entries. — tdelaney
– tdelaney, Commented Mar 25, 2017 at 18:07

tdelaney · Accepted Answer · 2017-03-25 18:25:08Z

1

The problem is that inner loop. You keep updating the same dict and re-adding it to data. data ends up with multiple references to the same dict and the dict has been updated to the last value in foobar.

Just copy before adding. Also, a note on good question writing... you want a runnable example when possible. Here I mock queryset so the thing runs.

class Mock:
    def __init__(self, id, created_at):
        self.id = id
        self.created_at = created_at

queryset = [Mock(1, '2017-01-01'), Mock(2, '2017-01-02')]

foobar=['foo','bar']
data = []
for q in queryset:
    row = {"id": q.id,
           "created_at": q.created_at}
    for f in foobar:
        row['index'] = f
        data.append(row.copy())
bulksave(data)

UPDATE

If you really want to confuse the next person to look at your code, collapse it all into

import itertools

class Mock:
    def __init__(self, id, created_at):
        self.id = id
        self.created_at = created_at

queryset = [Mock(1, '2017-01-01'), Mock(2, '2017-01-02')]

foobar=['foo','bar']

bulkupdate(list(dict(zip(('id', 'created_at', 'index'), (q.id, q.created_at, index)))
    for q, index in itertools.product(queryset, foobar)))

(dgg32's now-deleted post got me thinking...)

edited Mar 25, 2017 at 18:25

answered Mar 25, 2017 at 18:06

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

TomSanders Over a year ago

thnx for the help and the pointers. first question ever... never too old to learn :)

tdelaney Over a year ago

You wouldn't happen to be the SLC Tom Sanders?

Collectives™ on Stack Overflow

python: add one element to an existing dictionary from list loop

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related