How to minimize loop execution time in Python?

Question

I am trying to execute this code using for loop,

for i in fielding: # fielding has nearly 19000 records - dtype (list of dictionary)
   for j in player: # player has nearly 17000 records - dtype (list of dictionary)
       if i['id'] == j['id']:
          j['name'] = 'Luke'

So it takes longer time to run. Is there any alternative way to achieve this ? Thanks for your help.

what data types are fielding, player are they list, tuple, or what? — George Imerlishvili
– George Imerlishvili, Commented Apr 27, 2021 at 8:29
And what is supposed to happen with the outputs? Are you appending? Adding to something else? Or are you just wanting to update your dictionary? — brobertsp
– brobertsp, Commented Apr 27, 2021 at 8:31
But 'fielding' isn't a list of dictionaries, its just a dictionary correct? — brobertsp
– brobertsp, Commented Apr 27, 2021 at 8:34

madbird · Accepted Answer · 2021-04-27 08:48:26Z

6

Looks like you're trying to join data by an identifier (or any other fields). And that's the case when you may use dict or set to speedup id-lookups:

# Build a set of ids you need to update
fielding_ids = {f['id'] for f in fielding}

# Or use dict if you will need to access the records
# fielding_ids = {f['id']: f for f in fielding}

# Update records with corresponding ids from the second list
for p in player:
    if p['id'] in fielding_ids:
        p['name'] = 'Luke'

Python set() and dict() are hashmap-based containers, which are optimized for almost instant key presence checks.

answered Apr 27, 2021 at 8:48

madbird

1,3797 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

IODEV Over a year ago

@saifislam, feel free to do a test with both solutions and compare the execution time.

madbird Over a year ago

I think it's pretty close to amortized O(m+n), and there's no way to do it faster. Am I missing something?

IODEV Over a year ago

I meant @saifs original loop compared to your solution (using 19000*17000)..

Collectives™ on Stack Overflow

How to minimize loop execution time in Python?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related