Efficient Loop to Avoid Multiple Garbage Collect Calls for Android

Question

I have an Android app that allows the user to record data (such as accelerometer, latitude, longitude, etc.). There are a total of 9 of these fields, and the user can record for up to 10 minutes (3000 records per field). Therefore, a total of 27,000 data points can be collected. The user can also take pictures and videos to upload the SD Card as well.

When the user is done collecting data (or the 10 minutes are up), the data is stored in a String, which is later uploaded to the SD Card as a .csv file. The problem though is that it takes forever for the data to append to the string, due to the massive amount of garbage collecting (it appears to be approximately 5 collects or so per second!) The appending starts off fast, but seems to be slowing down as more and more as data is added.

This is the loop that causes the lag:

           for( i = 0 ; i < len2 ; i++ ) {
                data += accelX[i] + ", " + accelY[i] + ", " + accelZ[i] + 
                ", " + accelT[i] + ", " + latitu[i] + ", " + 
                longit[i] + ", " + orient[i] + ", " + 
                magneX[i] + ", " + magneY[i] + ", " + magneZ[i] + 
                ", " + millis[i] + "\n";

                partialProg = 100.0 * ( (double)(i+1) / (double)(len2));
                dia.setProgress((int) partialProg);

            }

data is just a String, and nothing is being newed, so I'm unsure of why GC is being called so often. My question is: what is the problem here, and/or how can I make this more efficient?

Why not use some kind of data storage structure (e.g. most sensibly a database)? Once each "round" of data is stored in the database it can be essentially "forgotten" until the end when you compile the CSV. — Thomas Clayson
– Thomas Clayson, Commented Jan 5, 2012 at 15:25
Consider using a PrintWriter in combination with a BufferedWriter and writing it directly to the file. If you still want to have the String in memory, I would recommend using a StringWriter. — Marcelo
– Marcelo, Commented Jan 5, 2012 at 15:26

amit · Accepted Answer · 2012-01-05 15:26:25Z

3

You are creating a lot of objects. every time you use the operator+, you actually create a new object, which is expansive [if you repeat it a lot of times]

You can make it more efficient by using a StringBuilder and append to it, and create the String when you are done.

for example:

sb.append(accelX[i]).append(',').append(accelY[i]).append(',').append(accelZ[i]);

[where sb is an instance of StringBuilder]

answered Jan 5, 2012 at 15:26

amit

179k27 gold badges245 silver badges348 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

seanhodges · Accepted Answer · 2012-01-05 16:00:47Z

1

You could use StringBuilder to concatenate the data:

StringBuilder sb = new StringBuilder();
for( i = 0 ; i < len2 ; i++ ) {
    sb.append(accelX[i]).append(", ");
    sb.append(accelY[i]).append(", ");
    sb.append(accelZ[i]).append(", ");
    sb.append(accelT[i]).append(", ");
    sb.append(latitu[i]).append(", ");
    sb.append(longit[i]).append(", ");
    sb.append(orient[i]).append(", ");
    sb.append(magneX[i]).append(", ");
    sb.append(magneY[i]).append(", ");
    sb.append(magneZ[i]).append(", ");
    sb.append(millis[i]).append("\n");
}

StringBuilder is inherently faster for building long strings.

It also avoids allocating so many String objects into the heap; since each += operator is creating a new String object rather than modifying the last one. This in turn leads to a large number of GC calls to clean up all the redundent String objects. See also: http://chaoticjava.com/posts/stringbuilder-vs-string/

As Marcelo points out; you may also find dealing with large amounts of data in memory may become problematic on low-spec Android devices, at which point you should consider appending the contents of your StringBuilder to a temporary file every X number of iterations to keep the footprint low. At the end of the process you can stream the file to whatever the destination is planned to be, or read segments of it back to memory on demand.

edited Jan 5, 2012 at 16:00

answered Jan 5, 2012 at 15:37

seanhodges

17.6k17 gold badges74 silver badges95 bronze badges

3 Comments

RokL Over a year ago

It is not true that each + operator creates a new String object, the very link you posted debunks that. One expression with + operators uses one StringBuilder.

Mxyk Over a year ago

All the other answers were helpful, but approved this one because of the nice explanation/full code example. Thank you, this is working.

seanhodges Over a year ago

U Mad: he is not executing one expression, he is appending to "data" on each loop. But you are right, under single assignment situations the compiler can optimise out the operators with a StringBuilder.

kosa · Accepted Answer · 2012-01-05 15:32:35Z

1

One improvement you may do is, StringBuffer to construct data, that way you don't you can avoid String construction and concatenation operations. For example:

StringBuffer buff = new StringBuffer();
buff.append(accelX[i]).append(...).apend(...)

edited Jan 5, 2012 at 15:32

answered Jan 5, 2012 at 15:26

kosa

66.7k15 gold badges134 silver badges170 bronze badges

1 Comment

amit Over a year ago

-1 for still creating a lot of String objects inside the append: accelX[i]+","+..... is also creating more String objects

RokL · Accepted Answer · 2012-01-05 15:50:27Z

+ operator on strings actually uses StringBuilder, thus you do perform at least two allocations per loop (new StringBuilder and then StringBuilder creates a string when .toString() is called on it to assign the result to data). Best way to tackle this is to create a StringBuilder in front of the loop.

StringBuilder buf = new StringBuilder();
String sep = ", ";
for( i = 0 ; i < len2 ; i++ ) {
    buf.append(accelX[i]).append(sep).append(accelY[i]).append(sep);
    buf.append(accelZ[i]).append(sep).append(accelT[i]).append(sep);
    buf.append(latitu[i]).append(sep).append(longit[i]).append(sep);
    buf.append(magneX[i]).append(sep).append(magneY[i]).append(sep);
    buf.append(magneZ[i]).append(sep).append(millis[i]).append('\n');
    partialProg = 100.0 * ( (double)(i+1) / (double)(len2));
    dia.setProgress((int) partialProg);
}

Collectives™ on Stack Overflow

Efficient Loop to Avoid Multiple Garbage Collect Calls for Android

4 Answers 4

Comments

3 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

3 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related