Reduce time for Loop over multiple objects to get value Django

Question

I have a Dataframe in which a column Code is increased significantly everyday and these codes are to be converted into object description for which I am doing something like the following:

    product = []
    beacon = []
    count = []
    c_start = time.time()
    for i, v in df["D Code"].iteritems():
        product.append(Product.objects.get(short_code=v[:2]).description)  #how to optimize this?
        beacon.append("RFID")
        count.append(v[-5:])
    c_end = time.time()
    print("D Code loop time ", c_end-c_start)

Now initially when the rows were less it used to work in no time but as the data increased the combined Database call for every code takes too much time. Is there any more efficient Django method to loop over a list and get the value?

The df['D Code]` looks something like this:

['TRRFF.1T22AD0029',
  'TRRFF.1T22AD0041',
  'TRRFF.1T22AD0009',
  'TRRFF.1T22AD0032',
  'TRRFF.1T22AD0028',
  'TRRFF.1T22AD0026',
  'TRRFF.1T22AD0040',
  'HTRFF.1T22AD0003',
  'TRRFF.1T22AD0048',
  'PPRFP.1T22AD0017',
  'TRRFF.1T22AD0047',
  'TRRFF.1T22AD0005',
  'TRRFF.1T22AD0033',
  'TRRFF.1T22AD0024',
  'TRRFF.1T22AD0042'],

lucutzu33 · Accepted Answer · 2022-05-30 09:15:55Z

1

You can create a lookup dict with just one query. Then use that dict to find your description.

description_dict = {}
for product in Product.objects.values('short_code', 'description'):
    description_dict[product['short_code'] = product['description']

for i, v in df["D Code"].iteritems():
        product.append(description_dict[v[:2]])
        ...

answered May 30, 2022 at 9:15

lucutzu33

3,7651 gold badge14 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reduce time for Loop over multiple objects to get value Django

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related