0

I've been pondering moving our current admin system over to a JS framework for a while and I tested out AngularJS today. I really like how powerful it is. I created a demo application (source: https://github.com/andyhmltn/portfolio-viewer) that has a list of 'items' and displays them in a paginated list that you can order/search in realtime.

The problem that I'm having is figuring out how I would replicate this kind of behaviour with a larger data set. Ideally, I want to have a table of items that's sortable/searchable and paginated that's all in realtime.

The part that concerns me is that this table will have 10,000+ records at least. At current, that's no problem as it's a PHP file that limits the query to the current page and appends any search options to the end. The demo above only has about 15-20 records in. I'm wondering how hard it would be to do the same thing with such a large amount of records without pulling all of them into one JSON request at once as it'll be incredibly slow.

Does anyone have any ideas?

Improve this question
3
  • Why not just synthesize 10000 records? (or just append the same 25 ones 400 times?) Then you'll see for yourself. And if that won't work you can even mix-and-match: a client side application with server-side paging and sorting. Commented Sep 10, 2013 at 11:30
  • Sorry, I fail to grasp the point: you want to download all the record to the client or not? Commented Sep 10, 2013 at 11:32
  • Preferably not. I'm just trying to think of an alternative at the moment Commented Sep 10, 2013 at 11:34

2 Answers 2

Reset to default
4

I'm used to handle large datasets in JavaScript, and I would suggest you to :

  • use pagination (either server-sided or client-sided, depending on the actual volume of your data, see below)
  • use Crossfilter.js to group your records and adopt a several-levels architecture in your GUI (records per month, double click, records per day for the clicked month, etc.)

An indicator I often use is the following :

rowsAmount x columnsAmount x dataManipulationsPerRow

Also, consider the fact that handling large datasets and displaying them are two very differents things.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

Indeed pulling so many rows in one request would be a killer. Fortunately Angular has the ng-grid component that can do server-side paging (among many other things). Instructions are provided in the given link.

Improve this answer

3 Comments

Thank you very much! I'll look into it and then see if it solves what I'm trying to achieve :)
I tried ng-grid in my project and found it to not be ready for prime time. At the time of my trial, at least, the grid didn't have footer-row (summary) capabilities. It is also implemented as a set of DIVs instead of a table. At first, this doesn't seem so bad, but for some reason, the text isn't selectable. Even if it were, without the semantic meaning of the table, users can't copy/paste into Excel. It made me sad, but it turned out that I was able to write my own full-featured, paging, sorting grid in Angular in less than 80 lines of code.
@BrianGenisio Text selection in ng-grid is disabled via CSS by default. Probably to make row selection a bit prettier. If you want it back turn it back on via CSS. user-select: text !important

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.