Fetching Large Data to render on Datatable - ReactJS

Question

I have a list of 20,000 rows in my database that I am fetching to my table. I have this 20,000 rows stored to a localStorage to improve speed. My issue is, when I am navigating to the page (which contains the 20,000 rows), it lags for few seconds before the page is loaded.

How can I let my page render and wait for the huge data to be fetched ?

PS: getItem function fetches the data and store in array items and stored to localStorage.setItem('my_items') .

Index.js

 state: {
     items:[],
     loading:true;
   }

componentDidMount()
{       

let items = JSON.parse(localStorage.getItem('my_items'));

if(items)
    {
      this.setState({
      items: items
    })      
     this.setState({ loading: false });
    }
    else
    {
        this.getItems();
    }
}





  return (

{loading &&
           <div className="d-flex justify-content-center loader-overlay">
              <CircularProgress />
           </div>
        }

       <MuiThemeProvider theme={this.getMuiTheme()}>

                <MUIDataTable

                    title={
                    }
                    data={this.state.items.map(item => {
                        return [
                           item.name,
                           item.type
                        ]

                     })}
                    columns={columns}
                    options={options}
                        />
            </MuiThemeProvider>   
        );

@AZ_, server side pagination ?? I am using data tables. So the data is already paginated on the table — Switz
– Switz, Commented Jul 22, 2019 at 6:48
I meant implement pagination when you are fetching the data from the table, already paginated in the table you mean sorted? — arizafar
– arizafar, Commented Jul 22, 2019 at 6:52
@Switz datatables comes with pagination, sorting. you should implement server-side processing: datatables.net/examples/data_sources/server_side — Exceptional NullPointer
– Exceptional NullPointer, Commented Jul 22, 2019 at 6:53
In terms of UX why would you want to load 20k rows in the first place? What would be the reason behind this? Maybe implement progressive loading (say with useEffect) as the user scrolls down the page. — Intellidroid
– Intellidroid, Commented Jul 22, 2019 at 6:54

Tomas Bruckner · Accepted Answer · 2019-07-22 07:02:09Z

2

This problem is usually solved by using server side pagination. It should work something like this:

Open page with data table and you fetch first 20 items (or whatever size of a page is). You send /api/tableData?offset=0&limit=20. This will only return first 20 items.
User clicks on next page, you fetch the second page using /api/tableData?offset=20&limit=20. This will return items with id 21-40.
User clicks on next page, you fetch the third page using /api/tableData?offset=20&limit=20

If you are using mui-datatables NPM package, it has option serverSide and it helps you to do it easily. Chjeck Remote Data section in the documentation of MUI datatables package https://www.npmjs.com/package/mui-datatables#remote-data.

But if you are sure about your use case and you don't want to do server side pagination, you can use Web Worker. The problem in your application is that JSON.parse for such a huge collection takes way too long. With Web Worker, you can start another thread that will not block the UI thread. In the worker thread, you will start parsing the collection and when you finish parsing, you will send the data back to the UI thread. Web workers were designed for exactly this use case. You can learn about it more here: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers

answered Jul 22, 2019 at 7:02

Tomas Bruckner

7282 gold badges10 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Switz Over a year ago

With this method, when the user is searching for an item, the search will be limited

Tomas Bruckner Over a year ago

If you want to use filtering, you should also do it server side. So the API call would change to something like /api/tableData?offset=0&limit=20&search=foo&search-column=name if you want to only have first 20 items that contain name foo. This is just an example, you can implement it however you like, but the principle is always the same. Do sorting, filtering and pagination on the server (if you are using mysql, you will use limit, offset and order by).

Collectives™ on Stack Overflow

Fetching Large Data to render on Datatable - ReactJS

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related