DNS stops responding

[rayout]

What is Happening

DNS stops responding at random times. It just freezes.
In this case, it can work for quite a long time before freezing

Specs

• Docker version 26.1.3, build b72abbb
• DPS Version:
  • defreitas/dns-proxy-server:3.24.0-snapshotproxy-server --version`
• OS: [e.g. Ubuntu 24.04]

[rayout]

Today it froze again

[mageddo]

Hey @rayout I will need the full log to debug what is happening to cause this behavior, can you share it? docker logs ${CONTAINER_ID} &> logs.log, please enable TRACE log level to give more details.

Another question: From which version did you notice this behavior?

[mageddo]

I will keep using DPS to see if I also get the issue

[dmekhov]

Sometimes (rarely) I also face the same (maybe different) issue (dns service stops answering) .

I have that in my logs (at 3.24.0 version):

Exception in thread "dnsjava NIO selector" java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.

[mageddo]

@dmekhov @rayout it can be related, DPS default heap size is set to 10m. You can test if increase the value fixes the issue for you by running

$ docker run  defreitas/dns-proxy-server:3.24.0-snapshot -XX:MaxHeapSize=50m -XX:MaxNewSize=10m

[dmekhov]

You can test if increase the value fixes the issue for you by running

$ docker run  defreitas/dns-proxy-server:3.24.0-snapshot -XX:MaxHeapSize=50m -XX:MaxNewSize=10m

Can I use env variables to configure it? (JAVA_OPTS, JVM_OPTS, etc ?)

I'm using docker compose setup.

(for now I set it via the command property and will check if it helps)

[mageddo]

I'm afraid you can't use the JVM env to configure native image binaries, but you can use the command option at the docker-compose file

services:
  dps:
    image: defreitas/dns-proxy-server:3.24.0-snapshot
    command: -XX:MaxHeapSize=50m -XX:MaxNewSize=10m

[dmekhov]

but you can use the command option at the docker-compose file

Yes, thanks, I use it now.

Ok, I'll be watching the result (but this error didn't happen often for me, so it can take a while)

[mageddo]

DPS got stuck today for me, I also got java.lang.OutOfMemoryError errr at the logs.

logs.txt

[mageddo]

FYI: Just released DPS 3.25.0, it increases resources utilization optimization, maybe it can fix the issue without the need of increasing the heap size.

[mageddo]

I reproduced the freezing scenario, reported by @rayout , it is different from the reported by @dmekhov , they are two different root causes causing the same behavior:

OutOfMemoryError

Scenario

When receiving a high number of requests considering the actual memory limits set, sometimes the heap exceed the size causing DPS freezing.

Solution

Optimizations were made at #436 version: 3.25.1

Increase Heap Size

services:
  dps:
    image: defreitas/dns-proxy-server:3.24.0-snapshot
    command: -XX:MaxHeapSize=50m -XX:MaxNewSize=10m

docker run  defreitas/dns-proxy-server:3.24.0-snapshot -XX:MaxHeapSize=50m -XX:MaxNewSize=10m

Random Freezing due deadlock

When receiving a high number of concurrent requests, the DPS cache can cause a deadlock, eventually locking all it's threads and freezing DPS

Solution

Fixes was made at #522, version 3.25.2

[mageddo]

A could have optimization will also be made at #524