Dealing with large Strings/ Arrays?

Question

I need to parse a lot of data. When I mean a lot, I'm talking around 5,000 - 10,000 characters. Right now, my code works with 285ish pieces of data. I'm using the Arduino prototyping platform. Here's my loop() in my sketch:

void loop() {

    if (client.available()) {
        char inChar = client.read();
        currentLine += inChar;
        if (inChar == '\n') { currentLine = ""; }

        if (currentLine.endsWith("[start]")) {
            readingData = true;
            theData = "";
        }

        if (readingData) {
            if (inChar != '[') {
                 theData += inChar;
                 //Serial.println("something!");
            }
            else {
                readingData = false;
                int count = theData.length()-0;
                theData = theData.substring(1, count);
                Serial.println(theData);
                doAction(100,count,theData);
                client.stop();
            }
        }
    }

    if (!client.connected()) {
        Serial.println();
        Serial.println("disconnecting.");
        client.stop();
        for(;;)
            ;
    }
}

Should I split it up into 20+ strings and put them in an array? I'm not sure if my 2KB of RAM will be able to handle that.

Where do your 10000 characters come from? This is unlikely to even fit in the AVR's memory, IIRC. — Bojangles
– Bojangles, Commented Dec 20, 2011 at 17:41
@JamWaffles: From a remote URL that will turn on/off a sequence of LEDs on my breadboard. — iosfreak
– iosfreak, Commented Dec 20, 2011 at 17:44
I can't understand how you need 10k characters to control some LEDs. As Basile says, the AVR doesn't have enough RAM, so you'll need to split your string up into chunks and send them in, say, 1KB parts. — Bojangles
– Bojangles, Commented Dec 20, 2011 at 17:46
Eventually it's going to control christmas lights. If a song is 300 beats (around 2-3 mins), then using my code it would be around 5,000 characters... — iosfreak
– iosfreak, Commented Dec 20, 2011 at 17:50

Peter Mortensen · Accepted Answer · 2011-12-22 19:27:30Z

4

With 2 kilobytes of RAM you can have no more than 2000 bytes of data (and in reality, significantly less, for example, perhaps only 1500 bytes, because of stack and global spaces).

If you need to process 20 kilobytes of data in memory, buy a bigger microcontroller.... (or program your Arduino to transmit the data to your PC which will process it).

edited Dec 22, 2011 at 19:27

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Dec 20, 2011 at 17:43

Basile Starynkevitch

231k18 gold badges323 silver badges578 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

iosfreak Over a year ago

Sending it to my computer isn't an option. It's going to run remotely using a arduino uno & ethernet shield. Can I expand my memory?

Bojangles Over a year ago

@phpnerd211 This is a good candidate for an XY Problem; solve the root cause instead of hacking a solution to your problem. You should send your data in ~1KB packets over your network from the computer.

Mooing Duck Over a year ago

@phpnerd211: You will have to parse and interpret bytes as you get them and not keep them in memory unless you absolutely have to. As it is, your code appears to only keep one line at a time in memory (perfect!) so RAM shouldn't be an issue. Though you may want to put the lines containing currentLine into an if (!readingData) {, which will save you a bit more memory (and time).

iosfreak Over a year ago

@Mooing Duck: Wow, I can't believe I didn't think of that. It all works fine and dandy now! Thanks.

Collectives™ on Stack Overflow

Dealing with large Strings/ Arrays?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related