2

In short I want to know how effective it is to use arraylists in Java to hold objects with lot of data in it. How long an arraylist can grow and is there any issues using arraylist to hold 2000+ customer details (objects) while at runtime? Does it hit the performance in any way? Or is there any better way to design app which needs to quickly access data?

I am developing a new module (customer lead tracker) for my small ERM application which also handles payroll details for a company. So far the data was not so huge, now with this module I am expecting the data base to grow fast and I will have to load 2000+ customer details from database to perform different data manipulations, updates.

I wanted some suggestion as to which approach would be better,

  1. Querying customer Database (100+ columns) and getting data to work with for each transaction. (A lot of seperate queries for each)

  2. Load each row into objects save it in an arraylist at the beginning of and use the list to work with each row when required. And save the objects (rows) at end of a transaction?

Sorry if I have asked a dump question, I am really a start up independent developer this may sound a bit awkward from an experienced developer's perspective.

Improve this question
3
  • I am curious to know, How this question implies to Java Swing ? Shall I start a new thread for this ? :-) Though +1 for the spirit in which the question is asked Commented Jun 4, 2013 at 6:16
  • sorry, This apps is developed using Java swing components through that is out of scope of this question.. Thanks Commented Jun 4, 2013 at 6:17
  • Also there are no dump questions....only answers can be dump;-) Commented Jun 4, 2013 at 6:26

2 Answers 2

Reset to default
2

Depends on how much memory you have.Querying DB for each and every transaction is not a good approach as well.A better approach would be load data into memory depending on your memory size and once you are done with it, remove it and fire next set of db queries.In thi way you can optimize memory as well as db queries.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

This is exactly I was thinking the right way would be.. Thank you.. What would be your suggestion in implementing this approach? I was thinking what is the best way to have this data in memory? (ArrayList gives me the flexibility to iterate through.. but not entirely sure about the drawbacks..)
ArrayList dont have any drawback except the size limitation.You ca also use HashMap which does not have this size limitation and you can use (primary key,object) as key value pair.
I just had a search around HashMap, seems interesting. Thank you for the inputs..
Hi Renjith, Hashmap is really cool I never learned about this. I just want to say a special thanks to you (from a Malayali, from this far away continent near south pole...), I didn't know it until I checked your linked in profile (We were almost neighbors as I am from Kayankulam.. :) ).. Cheers..
Hey....Tat was such a nice co-incidence:-)Feel free to ask any Programming/Java doubts u have:-)
|
2

  1. Any ArrayList can hold not much than 231-1 elements, due to int typed index of inner array.

  2. There is an approach called in memory Db which implies that you hold a lot of data in memory for gain fast access to it. But this approach also implies, that:

a. you have a lot of memory, available for holding all necessary data (it could be several tens of gigabytes);

b. you db implements compact form of data storage. It means that db will not contain ready java-objects, but fragments of byte-array data, from which you will contstruct objects on demand.

So, you need to reckon, how much memory you will need for all data that you want to load into memory and decide whether this approach eligible or not.

Improve this answer

1 Comment

Good to know about the second option you suggested. For the app in question I don't think the data would be so big in any near future, but would like to know how to implement this though.. do you have any resources which I can learn from? Thank you for the reply..

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.