Description
Version
- vert.x core: 3.5.0
- vert.x web: 3.5.0
Context
As per subject, I encountered a weird behavior where the default 404 message is returned (RoutingContextImpl.DEFAULT_404 => <html><body><h1>Resource not found</h1></body></html>) under a concurrent load.
In my application, I have only one regular, non-worker verticle which configures the HTTP server & a route. I deploy 16 instances of my verticle on 16 event loops. The aforementioned behavior happens when I have the endpoint route handlers chained fluently:
final Router router = Router.router(vertx);
router.route()
.handler(handler1)
.failureHandler(failureHandler1);
router.post("/blah")
.handler(handler2)
.handler(handler3)
.handler(handler4);
I DO NOT observe this behavior when my route is configured as per following code, which led me to believe that it has something to do with the fluent chaining of handlers. Maybe. I had a look at #740 and maybe this is related to Route thread-safety
final Router router = Router.router(vertx);
router.route().handler(handler1);
router.route().failureHandler(failureHandler1);
router.post("/blah").handler(handler2);
router.post("/blah").handler(handler3);
router.post("/blah").handler(handler4);
My application does not throw exceptions when this happens and I have a feeling that this is because Vertx cannot find the next route. By looking at the following code in RoutingContextImpl, the DEFAULT_404 is returned during the execution of checkHandleNoMatch() - when there is no next route nor failures.
@Override
public void next() {
if (!iterateNext()) {
checkHandleNoMatch();
}
}
Do you have a reproducer?
No
Steps to reproduce
I observe this behavior when I start applying a gradually increasing stepped request load at my service. At some stage, I start getting back 404 responses with <html><body><h1>Resource not found</h1></body></html>. When exactly the 404 start appearing is not deterministic, i.e.: I cannot say that after X requests or Y minutes, I always start seeing the first 404
For example, after applying request load using 50 concurrent clients for 20 minutes, out of the total of 60000 requests, I got 7 404 responses (<html><body><h1>Resource not found</h1></body></html>), with the rest of the responses returned 200 as expected. The number of 404s can vary each run. Also, the 404s are not returned one after another, they are returned in random order
Extra
- I tried to enable
TRACElogging, but I did not see anything useful in the logs (at least nothing has caught my attention assuspicious). By looking atRoutingContextImplBase.iterateNext(), I can see that it returnsfalsewhen there are no more routes, which causes the invocation ofcheckHandleNoMatch();