I am trying to read the bytes out of a file stored on Google Cloud Storage and send it in an HTTP POST request but i go this exception with larger files, the code is working with small files :
The exception is throwed on this line:
writer.write(blobstoreService.fetchData(new BlobKey(video.getBlobkey()), start, end));
This is my code:
URLConnection connection = new URL("http://myurl.com/").openConnection();
//set time out to infinite
connection.setConnectTimeout(0);
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
BufferedOutputStream writer = null;
OutputStream output = connection.getOutputStream();
writer = new BufferedOutputStream(output); // true = autoFlush, important!
//........
BlobInfoFactory blobInfoFactory = new BlobInfoFactory();
BlobInfo blobInfo = blobInfoFactory.loadBlobInfo(new BlobKey(video.getBlobkey()));
Long blobSize = blobInfo.getSize();
//max read on fetch
long maxReadFetch = 1015807;
//read the file in one time temporary
long i = 0;
long start = 0;
long end = 0;
while(i < blobSize) {
start = i;
end = i + maxReadFetch;
//determine end
if(end > blobSize) {
end = blobSize;
} else {
end--;
}
writer.write(blobstoreService.fetchData(new BlobKey(video.getBlobkey()), start, end));
i += maxReadFetch;
}
writer.flush(); // Important! Output cannot be closed. Close of writer will close output as well.
} finally {
if (input != null) try { input.close(); } catch (IOException logOrIgnore) {}
}
The stacktrace:
Uncaught exception from servlet java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2961) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:111) at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection$BufferingOutputStream.write(URLFetchServiceStreamHandler.java:460) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at java.io.FilterOutputStream.write(FilterOutputStream.java:97) at controller.CtrlWistia.upload(CtrlWistia.java:200) at controller.CtrlWistia.add(CtrlWistia.java:126) at controller.CtrlWistia.ajax(CtrlWistia.java:79) at controller.CtrlAjax.main(CtrlAjax.java:66) at vidaao.AjaxServlet.doPost(AjaxServlet.java:37) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176) at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145) at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92) at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
