Time-efficient way to append to existing header in existing CSV file in Java?

Question

I'd like to modify the first line or header, of an existing csv file by adding a string to the end of that line.

I've tried using BufferedWriter to do so, but I can only get it to append at the end of the file.

My working code:

      public static void writeStringtoCsvFile(String filePath, String input) throws IOException {
    PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(filePath, true)));
    out.append(input);
    out.close();
  }

It seems OpenCsv doesnt have an available method to append to an existing file either. The file I need to process is at least 160mb or 1+mil records in size, would the alternative method:

BufferedReader to read all the lines
Append whatever is needed to first line
BufferedWriter to write everything to a new file

be too slow? Is there a more elegant solution to this? Thanks!

Given that you don't want to append to the file but change its content throughout there's no way around rewriting all of it. — Voo
– Voo, Commented Jul 12, 2020 at 8:06
If you control the original creation of the files, then you might be able to optimise this process in some circumstances. If you can add some extra bytes to the first line, maybe trailing whitespace, when the file is created, then when it needs to be updated you can use a RandomAccessFile and overwrite some of the extra bytes. — tgdavies
– tgdavies, Commented Jul 12, 2020 at 12:06

Stephen C · Accepted Answer · 2020-07-12 09:13:40Z

1

File systems do not allow you to insert bytes into or delete bytes from the middle of a file without rewriting all of the bytes after the insertion / deletion point.

Since the file system doesn't support it, the Java file I/O APIs can't support it.

Since the Java file I/O APIs don't support it, neither can the OpenCvs APIs. (Or at least not without rewriting the file under the covers.)

So, the answer is that there isn't a more efficient way than reading writing all of the lines of the file.

Notes:

While it is technically possible to do an in-place rewrite of a file, it is safer (and possibly more efficient) to create a new file, write to it and rename it when the writing is completed.
Reading and writing a line at a time (using BufferedReader / BufferedWriter) is better than reading all lines into memory at the same time, then writing them all out. Especially if the files are large.

answered Jul 12, 2020 at 9:13

Stephen C

723k95 gold badges849 silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

BoredPanda Over a year ago

Could you explain on using BufferedWriter vs using the OpenCsv library to do the writing to file? Which would be faster?

Stephen C Over a year ago

I think that you should be able to do it faster using BufferedWriter, since you can avoid the overheads of parsing and formatting the lines of the CSV file.

BoredPanda Over a year ago

I tried the method i showed above, and in fact it seems to append to the same last line of the file instead of creating a new line. If only I could get this method to work for the FIRST line instead lol... and is this even expected??

Stephen C Over a year ago

Yes. It is expected. The "working" code in your question is incorrect. You need to implement this in the way I suggest in my answer. Read lines from an reader and write them to a writer ... with special handling for first (header) line.

Collectives™ on Stack Overflow

Time-efficient way to append to existing header in existing CSV file in Java?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related