Upgrade to Reactor 2.0.7 in order to fix memory leak

[daveburnsLIT]

Version : Spring Boot 1.3.0 RC1

There is a memory leak in the Stomp Relay Broker with active open stomp websocket connections. We are using a rabbitMQ backend stomp Broker. We initial noticed the memory usage pattern after running a long 10 hour endurance test.

To replicate the issue we setup a test to connect 2,000 websocket stomp connections to a RabbitMQ WebSocketMessageBroker using the Spring WebSocketStompClient. After monitoring the JVM for around 1 hour the server will either run out of memory or become unresponsive due to excessive garbage collection, then drop the connections. Profiling the JVM over this period shows a leak probably in either the reactor or netty libraries (although, afraid we’re no experts on memory leaks).

The attached sample application (based on the Spring Websocket getting started guide for the server) demonstrates the issue. The broker/server and Java Websocket clients were executed on separate machines during the test run. The WebsocketTest starts the client websocket connections.

Included in the sample app repo is the out of memory heap dump generated after a one hour test run. Also, the attached VisualVM screen shot shows the memory usage pattern where old gen memory accumulates even after all 2000 websocket clients are connected at the 6:10 marker.

[snicoll]

ping @rstoyanchev @smaldini

[rstoyanchev]

When you say memory leak, can you describe the memory usage pattern in more detail? How much have is allocated to the JVM, how much is used after the clients are connected, does it stay that way, or does it grow? Or is it perhaps at or near max memory and just happens to fall over after 1 hour? Have you tried giving it more memory than 500MB?

[daveburnsLIT]

Fast reply! Yeah, we were running our app in a cloud foundry env with 3GB initially then noticed via monitoring that the old gen mem was accumulating. We built the stripped down simple sample app and gave it less memory to speed up the out of memory error. Basically after about 10 minutes you have 2,000 connected clients and you see a saw-tooth type memory pattern on the heap useage. The problem is after the GC runs there's always a small piece of old gen not reclaimed, after long enough you see the "tooth" of the saw tooth get so small the CPU is used up doing garbage collection and we out of memory. Interestingly if we drop the socket connections from the client and then connect again we can run the test for longer, so the connection close seems to free the memory.

[stuartgreenlees]

I've uploaded the heap dump to the sample project repo.

[rstoyanchev]

Thanks is this a difference in behavior as a result of an upgrade, i.e. changing versions? Was it running okay before?

Profiling the JVM over this period shows a leak probably in either the reactor or netty libraries (although, afraid we’re no experts on memory leaks).

@smaldini, any chance you could have an initial look? I'm tied up currently with RFD-related fallout.

[stuartgreenlees]

We haven't had websockets running before. This is a new feature we are adding to our app.

[daveburnsLIT]

Did notice this behaviour on the M5 release also but saw there was a reactor perf fix in RC1, hoped that might have fixed this issue too. As Stuart says, new feature in our app so can't say when it crept in.

[daveburnsLIT]

We just ran our sample application against a Spring Boot 1.2.7 version of the server and got the following memory usage, very different pattern (looks to us like there's no leak in this version).

I suppose the main difference in libraries is the 1.3.0 release needs the io.projectreactor dependency whereas 1.2.7 uses the older org.projectreactor. (current Spring docs still state org.projectreactor, needs updated )

[smaldini]

Sorry for the late response, let me have a look, might be an issue with the use of Pooled ByteBuf factory from Netty in 2.0.6.

[smaldini]

Hey again, can you try io.projectreactor:reactor-net:2.0.7.BUILD-SNAPSHOT anytime on your side, I'm evaluating further deep, but I pushed a quick default behavior fix to that allocator in the meantime.

[smaldini]

Another comment (sorry!). The graph looks somehow fine from a certain perspective, with Bytes pooling, after a warmup period (incoming traffic, including heartbeats) it might reach its max potential and have pre-allocated all the room it needed. That means the cap we see in the end should tend to stay here (or may change again down or up to expand/shrink the bytes heap cache). It tends to give a more predictable memory model for sizable traffic volume. However it does seem strange the GC is that intensively used, and I would expect it to be worse in the older version without Bytes pooling or in the snapshot.

[daveburnsLIT]

Sorry Stephane, in the first graph we reach steady state (2000 connections) just after the 6:10 marker, if left long enough we do get an out of memory, watching the mem usage over time shows netty and reactor class instances continually growing even after GC. The second graph above seems to better represent what we would expect to see in steady state i.e. the same 2000 connections doing a ping/pong every 10 seconds.

[daveburnsLIT]

Oh, will setup that test on the 4.0.7 snapshot now and post back

[smaldini]

2.0.7.BUILD-SNAPSHOT for reactor-net only. Thanks a lot for your help, I'm still investigating tho as we might not be lucky on that quick check.