Storing large objects/data on disk to free browser memory in JavaScript

Question

I have a situation where I am dealing with very large files and trying to encrypt them. I am running into a problem in the browser where the browser is running out of memory and so is crashing.

I was hoping to try and offload some of the larger objects/data to a local cache on disk to take these out of memory to help with this.

I've seen the CacheAPI which I thought may help, but this seems to only handle storing requests, whereas I am really dealing with arrays of bytes and objects that I want to store somewhere.

https://web.dev/cache-api-quick-guide/

Is there anything else I can use to support this, I need to support modern browsers so not IE.

Maybe another approach would be to ask if you really need to handle objects and arrays so large that they overflow the computer's memory? Isn't streaming a solution? Chunking? — Jeremy Thille
– Jeremy Thille, Commented Jun 2, 2020 at 10:46
I am chunking before and after the encryption, the issue I have is with the encryption as I can't encrypt in chunks and send up to the server before I go to the next chunk I need to encrypt the whole thing at once. — user3284707
– user3284707, Commented Jun 2, 2020 at 10:48

grzim · Accepted Answer · 2020-06-02 11:00:09Z

2

You can try indexedDB - it supports the data you need.

https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API

localStorage size is 5000KB only, when it comes to IndexedDB you are limited by the disk space.

answered Jun 2, 2020 at 11:00

grzim

6044 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user3284707 Over a year ago

Ahh this sounds like it could be perfect... I will give it a go this afternoon

grzim Over a year ago

let me know if it helps:)

user3284707 Over a year ago

It didn't work out, maybe it was me but I couldn't get it to work with large byte arrays, it just kept crashing the browser every time I tried to add a row

Collectives™ on Stack Overflow

Storing large objects/data on disk to free browser memory in JavaScript

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related