Optimize Database Query Rails

Question

so I'm working on a Rails side project with a few friends of mine that requires a lot of seed data to be loaded. I have the seed data in a JSON document, which now contains just under 3200 JSON objects, each with the same 3 field names.

I've parsed and seeded this data in Rails, creating a new record for every JSON object. I should note that the object has a few has_many associations.

The issue is, after I've seeded the database with the JSON file, when I'm attempting to load all 3200 records into the page (using just a plain ModelName.all.each block), the load time is ridiculous...about 17826ms (17.8 seconds).

According to the console, it says (Views: 16252.4ms | ActiveRecord 1522.9ms). I've looked into implementing eager loading and pre-loading (and I still am working on it).

My question is, how can I optimize this further? I've tried ModelName.preload(:associationName) for pre-loading, but this doesn't seem to change the repsonse time much at all (perhaps I'm doing it incorrectly). What boggles my mind is that 3200 records is not very much...why is this taking so long? Would incorporating something quicker like Redis be a viable option this early in the project?

can you paste sample model data and the query that is being executed when you say ModelName.all. also paste the relevant model code. — Aditya Shedge
– Aditya Shedge, Commented Dec 22, 2014 at 18:11
@tagCincy ModelName.all is in a controller variable which gets called in the view with each — CisBae
– CisBae, Commented Dec 22, 2014 at 18:27

Mohammad AbuShady · Accepted Answer · 2014-12-22 19:39:03Z

2

You probably have an n+1 issue, sure you fetch all objects in the controller, but in the view you're accessing some relations that requires extra lazy queries, to fix that you need to eager load the whole data, using includes keyword

@all_objects_with_associations = SomeObject.includes(:some_association).all

Another extra level would be using fragment caching, assuming you don't want to paginate ( if you are using rails 4 it's enabled by default ), all you need to do is adding a cache block on the top of the template, and all variables that you think would require a cache invalidation if changed

cache @all_objects_with_associations do

To get more information you can check the following links

Eager loading in rails
Fragment caching in rails

answered Dec 22, 2014 at 19:39

Mohammad AbuShady

43.3k11 gold badges81 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

humza · Accepted Answer · 2014-12-22 17:26:44Z

1

It's the view that truly killing the speed. Rendering many objects will of course take time. You should paginate your data and display a limited number of them at a time. That will take the strain off the database as well as the view.

Use "will_paginate" for pagination.

Alternatively, you could implement "vertical pagination" i.e. make an AJAX call once the page is scrolled down to the bottom to fetch and append more results.

Furthermore, I would take a look at the view and ensure that no database queries were being made through it. If they were, I'd try to eliminate them with eager-loading or moving them to the controller somehow i.e. sending the data already to the view and not having the view fetch data.

answered Dec 22, 2014 at 17:26

humza

7,7643 gold badges21 silver badges29 bronze badges

1 Comment

CisBae Over a year ago

That's a very good point, I was thinking about pagination and vertical lazy loading, thanks so much for the suggestion!

Sahil · Accepted Answer · 2014-12-22 19:01:32Z

1

Apart from pagination, you can fetch only the required fields from the database, which will improve your performance further.

Since, your views are consuming a major time slice, consider the following:

using partials, helper methods (to simplify view logic)
moving rendering to client side (send JSON from server side)
adding cache strategies

If you can post some details about your views, it will help us provide more customized help.

answered Dec 22, 2014 at 19:01

Sahil

5046 silver badges9 bronze badges

Collectives™ on Stack Overflow

Optimize Database Query Rails

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related