How to reduce the memory usage when encoding base64 string in c#?

Question

I am using the following code to retrieve a bitmap from a file located on my computer system:

Bitmap shellThumb = sf.Thumbnail.ExtraLargeBitmap;

where sf is a ShellFile.

I then use the following to convert the bitmap to base64:

public static string ToBase64String(this Bitmap bmp, ImageFormat imageFormat)
    {
        string base64String = string.Empty;


        MemoryStream memoryStream = new MemoryStream();
        bmp.Save(memoryStream, imageFormat);

        memoryStream.Position = 0;
        byte[] byteBuffer = memoryStream.ToArray();

        memoryStream.Close();

        base64String = Convert.ToBase64String(byteBuffer);
        byteBuffer = null;

        return base64String;
    }

I then call this method using the following:

string base64ImageAndTag = shellThumb.ToBase64String(ImageFormat.Png);

I then use JSON.Net to write this string as well as other properties of the file to a JSON file to be used in another project - there is no alternative to JSON, it has to be used.

string json = JsonConvert.SerializeObject(files.ToArray(), Formatting.Indented, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });

where files is a List.

I have to read in a thumbnail for each file in a specified folder.

From reading in the files, I get an OutOfMemory Exception and the conversion is the cause of this while writing to the JSON file.

Is there a way to better encode the bitmap so that the exception is avoided? I have searched for solutions for this problem, but am not sure how/where to apply any possible solutions to make this work.

Many thanks!!

Do you know what actually causes the out-of-memory exception? 'Worse'-case scenario is to stream (via base64, CryptoStream with custom base64-ICryptoTransform might help) the images to file. There is no need to keep the entire file (including a lot of images?) in memory at the same time. — Caramiriel
– Caramiriel, Commented Mar 30, 2014 at 20:55
Hi @Caramiriel, thanks for your comment! The exception happens while its trying to write to the json file. I know it must be the base64 string because only after I added it in the exception occurred. The string is an attribute of a file object, so it needs to be written with its file object's attributes - what do you mean exactly by keep the entire file and images in memory at the same time? How would I address that? — sim1
– sim1, Commented Mar 30, 2014 at 21:27
Well, I don't know if there is such thing (haven't worked with it), but for XML there is XmlWriter that writes directly to a stream. This will allow you to write each element/attribute separately. An example for your case: when trying to serialize the second image, the first image is already on disk (not in memory anymore). Perhaps there is something like that for JSON as well. — Caramiriel
– Caramiriel, Commented Mar 31, 2014 at 7:04

Mike Zboray · Accepted Answer · 2014-03-30 23:45:20Z

One way to reduce the memory footprint is to serialize a single object at a time and stream them into the output rather than trying to generate all of the objects (and their Base64 strings) in-memory upfront. You would use the JsonSerializer and JsonTextWriter classes in Json.Net to stream the image data one at a time. Below is a short example comparing the two ways of serializing. You would probably replace the StringWriter with a StreamWriter pointed at an output file. Inside GenerateImageFile is where you would call your ToBase64String method.

class ImageFile
{
    public string Name { get; set; }
    public string Base64Image { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        var files = new[] { "test.bmp", "test2.bmp" };

        // generate all the objects and then serialize.
        var imageFiles = files.Select(GenerateImageFile);
        var serialized = JsonConvert.SerializeObject(imageFiles.ToArray(), Formatting.Indented, new JsonSerializerSettings() { NullValueHandling = NullValueHandling.Ignore });
        Console.WriteLine(serialized);

        // generate objects one at a time.
        // use JsonSerialzer/JsonTextWriter to "stream" the objects
        var s = JsonSerializer.Create(new JsonSerializerSettings() { NullValueHandling = NullValueHandling.Ignore });
        var strWriter = new StringWriter();
        using (var writer = new JsonTextWriter(strWriter) { Formatting = Formatting.Indented })
        {
            writer.WriteStartArray();
            foreach(var file in files)
            {
                var imageFile = GenerateImageFile(file);
                s.Serialize(writer, imageFile);
            }
            writer.WriteEndArray();
        }
        Console.WriteLine(strWriter.GetStringBuilder());
    }

    private static ImageFile GenerateImageFile(string fileName)
    {
        return new ImageFile() { Name = fileName, Base64Image = fileName + fileName };
    }
}

oleksii · Accepted Answer · 2014-03-30 20:38:36Z

0

You would want to compress it. If you want to preserve the bitmap format you can compress image with gzip (or any other better suited compressor). This will also be lossless compression, you don't lose any quality.

If you can convert it to jpeg, that will probably give you higher compression rate. Be careful though, this compression results in quality loss.

answered Mar 30, 2014 at 20:38

oleksii

36.1k16 gold badges98 silver badges173 bronze badges

2 Comments

sim1 Over a year ago

Hi @oleksii, thanks for your answer. Can I compress the bitmap programmatically? How would I do that??

oleksii Over a year ago

@user2651192, certainly you can. Bitmap is a series of bytes. You have already converted it to memory stream, you can then compress it. See this link on how to compress/decompress stream with gzip (in that case it's filestream, but it doesn't matter - any stream will work).

Collectives™ on Stack Overflow

How to reduce the memory usage when encoding base64 string in c#?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related