How to handle HTTP/2 GOAWAY with java.net.HttpClient?

Question

I am trying to continuously send GET and POST requests to a REST API every few minutes. The issue is that after exactly 1000 requests I receive a GOAWAY frame (and an IOException):

The GOAWAY frame (type=0x7) is used to initiate shutdown of a connection or to signal serious error conditions.
_{§ 6.8, RFC 7540}

I did a fair bit of research and found that not only is 1000 requests nginx's default maximum, Cloudfront (related Chromium issue) and Discord also exhibit the same behavior.

I tried to reproduce this problem with a local nginx server with the default HTTP/2 configuration:

server {
    listen 443 http2 ssl;
    http2_max_requests 1000;
    ...
}

var client = HttpClient.newBuilder()
        .version(HttpClient.Version.HTTP_2)
        .build();

for (var i = 0; i < 1100; i++) {
    var url = URI.create(String.format("https://localhost/images/test%d.jpg", i));

    var request = HttpRequest.newBuilder().uri(url).build();

    client.send(request, HttpResponse.BodyHandlers.discarding());
    System.out.printf("Image %d processed%n", i);
}

And after approximately 1000 requests, I get a GOAWAY error as expected:

...
Image 998 processed
Exception in thread "main" java.io.IOException: /127.0.0.1:49259: GOAWAY received

My first thought would be to check if the exception message contains the string "GOAWAY" and then retry the request accordingly:

try {
    client.send(request, HttpResponse.BodyHandlers.discarding());
} catch (IOException e) {
    if (e.getMessage().contains("GOAWAY")) {
        client.send(request, HttpResponse.BodyHandlers.discarding());
    } else throw e;
}

My issue with this approach is that the string comparison seems like it may be fragile. Additionally, since all I have is an IOException with a message, I can't differentiate between GOAWAY frames with a genuine error code (in which case I should probably stop sending requests) and those with NO_ERROR (in which case I could probably retry the request).

How should I correctly deal with/handle GOAWAY errors (apart from using HTTP/1.1 instead)?

sbordet · Accepted Answer · 2024-06-05 19:13:58Z

37

A server is entitled to close connections at any time, for any reason.

In the HTTP/2 GOAWAY frame there is the indication of what was the last stream processed by the server, so the client can know what stream needs to be resent when a connection is closed.

Unfortunately, the lastStreamId is not surfaced in java.net.http.HttpClient, so there is no way to know it and take appropriate actions.

Your alternatives could be:

to use other clients that support surfacing the lastStreamId
use a lower level HTTP/2 client where you will have the GOAWAY frame available and therefore access to the lastStreamId

[Disclaimer, I am the Jetty HTTP/2 implementer]
Jetty supports a lower level HTTP/2 client that you can use for your use case - you may want to give it a try. You can find an example of how to use Jetty's HTTP2Client here.

edited Jun 5, 2024 at 19:13

answered Mar 10, 2019 at 20:58

sbordet

18.9k3 gold badges57 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Govinda Sakhare Over a year ago

are the streams processed in any particular order? e.g. if lastStreamId=2

sbordet Over a year ago

Stream processing order is undefined. Implementations can choose whatever suits them best.

Dmitry Zinkevich Over a year ago

If you set "jdk.httpclient.HttpClient.log" system property to "requests,channel,headers,errors,frames:all" for example, then you will be able to see the stream id

Rafael Mar 27 at 13:21

Just for additional information, this is a reported bug on JDK11 and 17. You can track it here bugs.openjdk.org/browse/JDK-8335181

Collectives™ on Stack Overflow

How to handle HTTP/2 GOAWAY with java.net.HttpClient?

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related