Efficient Way of Loading Huge Amount of Data on PHP Web Page

Question

I am using code igniter(PHP) n MYSQL .I am extracting huge amount of data (around 60 thousand records from XML) into my mysql database.Then i am trying to load this data onto my web pages.I am successfully inserting data into my database using insert batch method of code igniter .But when I try to retrieve these records,it takes ages to load and display on the web page.

The approach I am using to insert data:

  $data = array(array('title' => 'My title' ,
  'name' => 'My Name' ,
  'date' => 'My date' ),
   array('title' =>'Another title','name'=> 'Another Name','date' =>'Another date'));

   $this->db->insert_batch('mytable', $data);

I am simply using select statement to retrieve records from single table.

My question :

How can I retrieve huge amount of records efficiently without having delays in page loading and display??

Mitch Satchwell · Accepted Answer · 2012-12-11 10:20:19Z

5

You should look into pagination unless there is a very good reason to display 60,000 records to the user in one go.

CodeIgniter provides a pagination class, see documentation for details on how to use it along with example code:

http://ellislab.com/codeigniter/user-guide/libraries/pagination.html

If you need more guidance on how to use this than provided by the documentation above, the following tutorial may help:

http://phpmaster.com/pagination-with-codeigniter/

edited Dec 11, 2012 at 10:20

answered Dec 11, 2012 at 10:12

Mitch Satchwell

4,8302 gold badges27 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Bilal Khalid Over a year ago

@MitchS.i have got no problem with pagination and it is working fine if i test it with small amount of data.The problem is : I can't load huge amount of data and i can't display data onto the browser as it is is taking ages to read data from mysql.Any idea??

Mitch Satchwell Over a year ago

If you use pagination, you won't need to read and display huge amounts of data - you display a set amount per page instead and only need to load that.

Germann Arlington · Accepted Answer · 2012-12-11 10:17:38Z

1

If for some VERY GOOD reason you have to display ALL records together than you may want to looking into flushing the buffer periodically (every 100 records or so). Otherwise see MitchS's answer

answered Dec 11, 2012 at 10:17

Germann Arlington

3,3532 gold badges19 silver badges21 bronze badges

Collectives™ on Stack Overflow

Efficient Way of Loading Huge Amount of Data on PHP Web Page

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related