1

I am trying out Parse for a possible backend for my app but I have a concern when dealing with large files that I want to upload to it.

in the documentation is says I need to convert a file to byte array and pass that into the ParseFile like so

byte[] data = "Working at Parse is great!".getBytes();
ParseFile file = new ParseFile("resume.txt", data);

however a large file can obviously throw an OutOfMemoryException here since loading the file into a byte array would load the whole thing into memory.

So my question is how would I go about uploading a large file to Parse while avoiding getting OOME?

Improve this question

3 Answers 3

Reset to default
1

It is very likely your users will end up crashing the app if they have full control of the upload process since many users don't have a solid understanding of file sizes. I noticed that parse has a nice wrapper class in their API for iOS (PFFile) which is limited to 10 MB, it's a pity they haven't implemented the same for android.

I think you are going to have to do this manually using their REST API. Have a look at their REST API, more specifically the files endpoint. You can easily use an HttpURLConnection in conjunction with a FileInputStream which gives you more flexibility over the upload process using streams. I'm usually more comfortable doing stuff manually rather than using a wrapper that's not exactly clear what's going on behind the scenes, but if you are not, I can post a minimalistic example.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

You would think this would be more of a priority on android since android has hard memory limit where iOS is more lenient on how much you can put into memory. anyway I will have a look at this
@tyczj you're exactly right, not to mention that the android ecosystem has been growing at the speed of light. The user base and the community of developers is massive, probably parse.com has a bit of catching up to do
0

One possible solution is to read in portions of the large file and upload each portion separately. You would need to do this in such a way that you can reassemble the file when you download it again.

Improve this answer

1 Comment

any possible example of this, like how would I know where to split the file so that it would be safe for me to load that section into memory? This would have to be safe for use on different types of devices too ie. ones with low specs
0

no reason that you cannot use the REST API and chunk the upload using normal http 1.1 headers.

You should read up on 'chunked-encoding' and figure out how to segment your overall byte-array in in android if its really that big.

In android, in the sender, you will have to segment stuff , explicitly managing io on both sides where there will be streams...

The output stream, you should be able to hook up directly to the http request objects "OUT STREAM". The input side is just a read of a smaller block of your input file so you can conserve memory.

You set up the http Connection

Connect it

get the OUTSTREAM for the http-connection-request and pass that to the streamCopy util that is handling your chunking.

HTTP req headers:

> Transfer-Encoding: chunked
> Expect: 100-continue

HTTP response headers from parse u should see...

something indicating a "100 continue" that its waiting for more data to be sent by the chunking process behind the outputstream attached to the request.

Remember to CLOSE streams/connections when done with the http POST.

Note - if you have big files you really should question what you are actually getting from the big size. All types of media files allow you to trim the size prior to the uploads.

Improve this answer

6 Comments

most files are going to be images however I want the user to have access to the full resolution image that was taken from the camera on the device and with supporting different screen sizes scaling the image before I upload isnt going to look good on tablets or even a TV screen that uses chromecast for example. I think the REST API is going to be my best option
well regarding Ccast, my exp is that you can cut still pics down to 1080 p and they look great on the TV Set. So, starting with a 12Mpix photo, cut it down to 1080p with bitmap processing then upload to parse. This saves at least 80% on a large jpg/png.
you can do the math. a 1080p TV set can only support so many pixels . most phone cams can produce a photo with a lot more than a tv can handle.
if you are only planning to show a small region at a time on the TV set then that would be diff story - quite a complex thing to throw at Chromecast IMO.
at risk of dead horsing this... my exp with CC and parse and photos is that i can cut a 12M photo down to 250-350K then upload to parse then display on CC on TVSET and it looks very,very good.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.