-
Notifications
You must be signed in to change notification settings - Fork 21.6k
Description
System information
Geth version: 1.9.19
OS & Version: Ubuntu 18.04
Expected behaviour
DNS server should always send messages compliant to RFC 1035. In particular, DNS should not send messages greater then 512 bytes over UDP.
Actual behaviour
Geth truncates DNS packets while discovering new peers. This causes in the background a loop of DNS queries, all doomed to fail.
DEBUG[08-16|20:19:10.158] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"This issue was also previously reported (#20713) but it seemed to be a pi-hole related issue so it was closed without a solution.
The problem is actually in the DNS server configuration:
- DNS server sends 526 bytes when responding to queries like this YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net.
- go DNS client follows RFC 1035 and it truncates the message to 512 bytes and then it goes in error while parsing
- go DNS client reports the error to geth
The first query for the root is always fine because message size is 497 bytes (check the last line MSG SIZE rcvd):
$ dig TXT all.mainnet.ethdisco.net
; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT all.mainnet.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49427
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;all.mainnet.ethdisco.net. IN TXT
;; ANSWER SECTION:
all.mainnet.ethdisco.net. 0 IN TXT "enrtree-root:v1 e=YT2KPIIE637MN76A2UTCOCZPFU l=FDXN3SN67NA5DKA4J2GOK7BVQI seq=1320 sig=frVtmMBzg6wg4Ddhv6Fx1MJaUlWOG7YwskBUcAL7dFNbwo7gMzTeumaJLpPenKH_CFkDkTWs5H6aKOTGSwZFjAE"
ns-224.awsdns-28.com. 0 IN A 205.251.192.224
ns-706.awsdns-24.net. 0 IN A 205.251.194.194
ns-1441.awsdns-52.org. 0 IN A 205.251.197.161
ns-1818.awsdns-35.co.uk. 0 IN A 205.251.199.26
ns-224.awsdns-28.com. 0 IN AAAA 2600:9000:5300:e000::1
ns-706.awsdns-24.net. 0 IN AAAA 2600:9000:5302:c200::1
;; Query time: 0 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:21:34 CEST 2020
;; MSG SIZE rcvd: 497The second query to the actual ENR tree instead raises the issue:
$ dig TXT YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net
; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37600
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net. IN TXT
;; ANSWER SECTION:
YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net. 0 IN TXT "enrtree-branch:EIJJYR2Y7HKP7GTBNSTTFDKISM,ZGIF73FX4IUMCY7SIF6AZ673EA,TE4PG4BD5CKZTCSQBCE45CMS2E,JZFQSASGA7S6GWOCXIEHNNY4NU,VT6KSVN7W3K4UF433DW2W6AVAU"
ns-224.awsdns-28.com. 0 IN A 205.251.192.224
ns-706.awsdns-24.net. 0 IN A 205.251.194.194
ns-1441.awsdns-52.org. 0 IN A 205.251.197.161
ns-1818.awsdns-35.co.uk. 0 IN A 205.251.199.26
ns-224.awsdns-28.com. 0 IN AAAA 2600:9000:5300:e000::1
ns-706.awsdns-24.net. 0 IN AAAA 2600:9000:5302:c200::1
;; Query time: 0 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:22:15 CEST 2020
;; MSG SIZE rcvd: 526How to solve the issue
Remove other records from the DNS response aside by TXT, in particular those not used like all As and AAAAs.
I didn't check all other networks, but Ropsten for example is already configured in a correct way:
$ dig TXT 3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net
; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> TXT 3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1631
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net. IN TXT
;; ANSWER SECTION:
3EAZQHDTFVH6TTYVLEGGHUT5TM.all.ropsten.ethdisco.net. 0 IN TXT "enrtree-branch:XX4U4ZKXUCOWBHPUOGISEAIDJE,4ZR6DWAKZTWUEMTCN47N3KY7YE,NHRHUEAR5OY4K4O6TCL5LUTCLM,PDPBUM23QXC32GCRPCWBZBA2NU,N7NYW2XJCJZE22MXJMGI4HVJO4,I6VIAEOHT4RJC54AA2HFP6QJSI,M7YILXPTHZ7PUJZERM5T72PABA,TC22I752VVE6Y4DYYWG2VOV6PY,OGDOAVH7BVKLSNWTCM42WP" "4IEA,CYV3DOHCVAL43SK5RYQCDIK6EI"
;; Query time: 23 msec
;; SERVER: 172.27.112.1#53(172.27.112.1)
;; WHEN: Sun Aug 16 19:21:11 CEST 2020
;; MSG SIZE rcvd: 418Steps to reproduce the behaviour
Start geth from scratch with verbosity level 4
Backtrace
$ geth --verbosity 4
INFO [08-16|20:19:09.860] Starting Geth on Ethereum mainnet...
INFO [08-16|20:19:09.860] Bumping default cache on mainnet provided=1024 updated=4096
DEBUG[08-16|20:19:09.860] Sanitizing Go's GC trigger percent=25
INFO [08-16|20:19:09.862] Maximum peer count ETH=50 LES=0 total=50
INFO [08-16|20:19:09.862] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
DEBUG[08-16|20:19:09.862] FS scan times list="28.5µs" set=700ns diff=800ns
INFO [08-16|20:19:09.862] Set global gas cap cap=25000000
INFO [08-16|20:19:09.862] Allocated trie memory caches clean=1023.00MiB dirty=1024.00MiB
[...]
DEBUG[08-16|20:19:10.158] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.159] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.160] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.161] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.162] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.163] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.165] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:10.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.157] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:11.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:12.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:12.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.156] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:13.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:14.157] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:14.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.156] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:15.823] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:16.156] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:16.490] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
DEBUG[08-16|20:19:17.156] Error in DNS random node sync tree=all.mainnet.ethdisco.net err="lookup YT2KPIIE637MN76A2UTCOCZPFU.all.mainnet.ethdisco.net on 172.27.112.1:53: cannot unmarshal DNS message"
[...]