Skip to content

Wrong DNS configuration for ethdisco.net domain causes a lot of background errors during DNS queries #21454

@Neurone

Description

@Neurone

System information

Geth version: 1.9.19
OS & Version: Ubuntu 18.04

Expected behaviour

DNS server should always send messages compliant to RFC 1035. In particular, DNS should not send messages greater then 512 bytes over UDP.

Actual behaviour

Geth truncates DNS packets while discovering new peers. This causes in the background a loop of DNS queries, all doomed to fail.

DEBUG[08-16|20:19:10.158] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"

This issue was also previously reported (#20713) but it seemed to be a pi-hole related issue so it was closed without a solution.

The problem is actually in the DNS server configuration:

  1. DNS server sends 526 bytes when responding to queries like this YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net.
  2. go DNS client follows RFC 1035 and it truncates the message to 512 bytes and then it goes in error while parsing
  3. go DNS client reports the error to geth

The first query for the root is always fine because message size is 497 bytes (check the last line MSG SIZE rcvd):

$ dig TXT all.mainnet.ethdisco.net                           

; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT all.mainnet.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49427
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;all.mainnet.ethdisco.net.      IN      TXT

;; ANSWER SECTION:
all.mainnet.ethdisco.net. 0     IN      TXT     "enrtree-root:v1 e=YT2KPIIE637MN76A2UTCOCZPFU l=FDXN3SN67NA5DKA4J2GOK7BVQI seq=1320 sig=frVtmMBzg6wg4Ddhv6Fx1MJaUlWOG7YwskBUcAL7dFNbwo7gMzTeumaJLpPenKH_CFkDkTWs5H6aKOTGSwZFjAE"
ns-224.awsdns-28.com.   0       IN      A       205.251.192.224
ns-706.awsdns-24.net.   0       IN      A       205.251.194.194
ns-1441.awsdns-52.org.  0       IN      A       205.251.197.161
ns-1818.awsdns-35.co.uk. 0      IN      A       205.251.199.26
ns-224.awsdns-28.com.   0       IN      AAAA    2600:9000:5300:e000::1
ns-706.awsdns-24.net.   0       IN      AAAA    2600:9000:5302:c200::1

;; Query time: 0 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:21:34 CEST 2020
;; MSG SIZE  rcvd: 497

The second query to the actual ENR tree instead raises the issue:

$ dig TXT YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net 

; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37600
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net. IN        TXT

;; ANSWER SECTION:
YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net. 0 IN TXT "enrtree-branch:EIJJYR2Y7HKP7GTBNSTTFDKISM,ZGIF73FX4IUMCY7SIF6AZ673EA,TE4PG4BD5CKZTCSQBCE45CMS2E,JZFQSASGA7S6GWOCXIEHNNY4NU,VT6KSVN7W3K4UF433DW2W6AVAU"
ns-224.awsdns-28.com.   0       IN      A       205.251.192.224
ns-706.awsdns-24.net.   0       IN      A       205.251.194.194
ns-1441.awsdns-52.org.  0       IN      A       205.251.197.161
ns-1818.awsdns-35.co.uk. 0      IN      A       205.251.199.26
ns-224.awsdns-28.com.   0       IN      AAAA    2600:9000:5300:e000::1
ns-706.awsdns-24.net.   0       IN      AAAA    2600:9000:5302:c200::1

;; Query time: 0 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:22:15 CEST 2020
;; MSG SIZE  rcvd: 526

How to solve the issue

Remove other records from the DNS response aside by TXT, in particular those not used like all As and AAAAs.

I didn't check all other networks, but Ropsten for example is already configured in a correct way:

$ dig TXT 3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net

; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT 3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1631
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net. IN        TXT

;; ANSWER SECTION:
3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net. 0 IN TXT "enrtree-branch:XX4U4ZKXUCOWBHPUOGISEAIDJE,4ZR6DWAKZTWUEMTCN47N3KY7YE,NHRHUEAR5OY4K4O6TCL5LUTCLM,PDPBUM23QXC32GCRPCWBZBA2NU,N7NYW2XJCJZE22MXJMGI4HVJO4,I6VIAEOHT4RJC54AA2HFP6QJSI,M7YILXPTHZ7PUJZERM5T72PABA,TC22I752VVE6Y4DYYWG2VOV6PY,OGDOAVH7BVKLSNWTCM42WP" "4IEA,CYV3DOHCVAL43SK5RYQCDIK6EI"

;; Query time: 23 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:21:11 CEST 2020
;; MSG SIZE  rcvd: 418

Steps to reproduce the behaviour

Start geth from scratch with verbosity level 4

Backtrace

$ geth --verbosity 4
INFO [08-16|20:19:09.860] Starting Geth on Ethereum mainnet... 
INFO [08-16|20:19:09.860] Bumping default cache on mainnet         provided=1024 updated=4096
DEBUG[08-16|20:19:09.860] Sanitizing Go's GC trigger               percent=25
INFO [08-16|20:19:09.862] Maximum peer count                       ETH=50 LES=0 total=50
INFO [08-16|20:19:09.862] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
DEBUG[08-16|20:19:09.862] FS scan times                            list="28.5µs" set=700ns diff=800ns
INFO [08-16|20:19:09.862] Set global gas cap                       cap=25000000
INFO [08-16|20:19:09.862] Allocated trie memory caches             clean=1023.00MiB dirty=1024.00MiB
[...]
DEBUG[08-16|20:19:10.158] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.159] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.160] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.161] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.162] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.163] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.165] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.157] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:12.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:12.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.156] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:14.157] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:14.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.156] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.823] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:16.156] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:16.490] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:17.156] Error in DNS random node sync            tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
[...]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions