I have a spring boot application which serves requests from different users. Some requests might take long time and eventually cause a timeout. If a user would make lot of such calls, the entire server might run out available http threads, all other users would be affected also.
To prevent such situation and allow other users to still use the system, I need a rate limiter to prevent the thread exhaustion. An incoming call should fail if already N threads are taken for a given user.
My implementation below uses a Filter class which accepts/fails incoming request based on internal user<->thread-count map.
This approach does the job, however it is not resilient for occasional request peaks. If a user sometimes makes a bit more calls than allowed, it gets HTTP 429 error IMMEDIATELY. Ideally the Filter should wait for few more seconds while retrying. Only if after certain time there are no free threads, an error should be thrown.
Are there simple ways to achieve this without too much advanced java concurrent code?
My current implementation looks as follows, it rejects the call immediately after reaching the limit.
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
String userName = null;
Integer counter = 0;
try {
HttpServletRequest httpRequest = (HttpServletRequest)request;
userName = getUsername(httpRequest);
boolean ok;
synchronized (lock) {
counter = counters.get(userName);
if (counter==null) {
counter = new Integer(0);
}
counter++;
counters.put(userName, counter);
ok = counter <= limit;
}
if (ok) {
chain.doFilter(request, response);
} else {
throw new RuntimeException("Too many calls");
}
} finally {
synchronized (lock) {
counter--;
counters.put(userName,counter);
}
}
}