Logging large string causes OutOfMemoryError

Question

My Spring Boot web application connects to many external services and it has a requirement to write all requests and responses to and from the services to log files.

I use Logback for the log engine. And following code is for printing the response message to log files.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

....

private static final Logger LOG = LoggerFactory.getLogger(RequestLoggingClientRequestInterceptor.class);

...

ClientHttpResponse response = execution.execute(request, body);
InputStream s = response.getBody();
String loggingResponseBody = new String(ByteStreams.toByteArray(s), Charset.forName("UTF-8"));
LOG.info("response status code: {}, response headers: {}, response body: {}",
            response.getStatusCode(),
            response.getHeaders(),
            loggingResponseBody);

This code has performance problem. It consumes high CPU, causes high latency and, in some case where the response message is very large, it causes an OutOfMemoryError in execution of new String(ByteStreams.toByteArray(s), Charset.forName("UTF-8"));

Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.lang.StringCoding.decode(StringCoding.java:215)
    at java.lang.String.<init>(String.java:463)
    at java.lang.String.<init>(String.java:515)
    at ...

Note that I have no specific requirement to decode the string to 'UTF-8', I just did so because the constructor of the String class suggests.

Please suggest how to improve the code to solve the performance problem. I have tried AsyncAppender, though it helps with the latency, I am stuck with the OutOfMemoryError.

Is this a service which collects and prints logs from all services ? How big is the response body that causes OOM ? — Aarish Ramesh
– Aarish Ramesh, Commented Sep 5, 2018 at 8:24

Joop Eggen · Accepted Answer · 2018-09-05 09:14:40Z

2

A ClientHttpResponse must be closed (as specified), which ByteStreams.toByteArray(s) does not (as specified).

try (InputStream s = response.getBody()) {
    LOG.info("response status code: {}, response headers: {}, response body:",
            response.getStatusCode(),
            response.getHeaders());
    String loggingResponseBody = new String(ByteStreams.toByteArray(s),
            StandardCharsets.UTF_8);
    LOG.info(loggingResponseBody); // One param means no format with {}
}

So it might just be a resource leak. The code seems brittle, as it must be certain that the response is UTF-8 text and not exceeding senseless megabytes.

edited Sep 5, 2018 at 9:14

answered Sep 5, 2018 at 8:30

Joop Eggen

110k8 gold badges89 silver badges142 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

asinkxcoswt Over a year ago

You are right, it's a resource leak problem. Thank you.

Collectives™ on Stack Overflow

Logging large string causes OutOfMemoryError

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related