You need to open the file in append mode:
SD.open("file.txt", O_RDWR | O_APPEND);
It depends on the file being there. If it fails to open because the file isn't there you can open it in "create" mode instead:
SD.open("file.txt", O_RDWR | O_CREAT);
For example:
// Try and append
File f = SD.open("file.txt", O_RDWR | O_APPEND);
if (!f) {
// It failed, so try and make a new file.
f = SD.open("file.txt", O_RDWR | O_CREAT);
if (!f) {
// It failed too, so give up.
Serial.println("Failed to open file.txt");
}
}