Getting OutOfMemory Exception while converting tif file into base64 encoding

Question

We have created console application in C#, where will read multi-page tif/tiff files, splitting pagewise and then convert into base64 encoding to upload the file in other target application (as this accepts only base64 encoding to upload the documents) and we are getting this Out of Memory Exception whenever file size exceeded more than 500 MB Exception of type 'System.OutOfMemoryException' was thrown

Exception at System.Convert.ToBase64String(Byte[] inArray, Int32 offset, Int32 length, Base64FormattingOptions options)
   at System.Convert.ToBase64String(Byte[] inArray)

Code Snippet:

Byte[] bytes = File.ReadAllBytes(filepath);
String base64stringofdocument = Convert.ToBase64String(bytes);

Above filepath refers > absolute path of the file

You will probably have to do some memory usage analysis with Visual Studio to find the root cause of this. — Matthew Watson
– Matthew Watson, Commented Aug 7, 2020 at 14:02
A single string object can't exceed 2^31-1 chars in length in .NET. Base64 encoding takes up at least 4/3 space relative to the input size. That gives you an upper theoretical file size boundary of ~805MB, although practically it's probably lower. You probably want to compress the input before converting to base64 — Mathias R. Jessen
– Mathias R. Jessen, Commented Aug 7, 2020 at 14:06
Have you considered streaming the data instead of reading the whole file into memory? — Charleh
– Charleh, Commented Aug 7, 2020 at 14:07
Project > Properties > Build tab, untick the "Prefer 32-bit" checkbox. You don't prefer it. — Hans Passant
– Hans Passant, Commented Aug 7, 2020 at 14:57
@Charleh Just refer above code snippet, converting into byte array from the file before passing into base64 encoding. How to improve this ? Pls help us — Sheik
– Sheik, Commented Aug 7, 2020 at 15:37

Jack A. · Accepted Answer · 2020-08-07 14:33:45Z

2

There is overhead involved in working with strings. For very large amounts of data, you are better off working with arrays or streams. In this case, you can start by rewriting your code to use Convert.ToBase64CharArray. So your code would change to something like this:

Byte[] bytes = File.ReadAllBytes(filePath);
// Compute the number of Base64 converted characters required (must be divisible by 4)
long charsRequired = (long)((4.0d/3.0d) * bytes.Length);
if (charsRequired % 4 != 0) {
    charsRequired += 4 - charsRequired % 4;
}
// Allocate buffer for characters, and write converted data into the array
Char[] chars = new Char[charsRequired];
Convert.ToBase64CharArray(bytes, 0, bytes.Length, chars, 0);

Then you can upload the chars array to your target application.

answered Aug 7, 2020 at 14:33

Jack A.

4,5531 gold badge28 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sheik Over a year ago

Hi Jack, Tried with the above suggestion but if convert char array again into string object, the same issue might occur as the target application accepts string object only for uploading a document. Is there any better way to handle large files. Pls share code snippet.

Jack A. Over a year ago

@Sheik It seems unlikely that your target application can only accept .NET strings. If you edit your post to include the code used to upload to your target application, it may be possible to comment further.

Sheik Over a year ago

After the above code lines, We will just pass encoded base64 string into Webservice call of the target application. Nothing much after this step.

Collectives™ on Stack Overflow

Getting OutOfMemory Exception while converting tif file into base64 encoding

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related