DNS server mixes AUTHORITY/ADDITIONAL section into ANSWER section while responding to queries

[Neurone]

Environment

Windows build number: Microsoft Windows [Version 10.0.19041.450]
Your Distribution version: Ubuntu 18.04
Whether the issue is on WSL 2 and/or WSL 1: WSL2 Linux version 4.19.104-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Feb 19 06:37:35 UTC 2020

Steps to reproduce

Query the TXT record of a domain, for example:

➜  ~ dig txt ultradns.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> txt ultradns.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17666
;; flags: qr rd ad; QUERY: 1, ANSWER: 22, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;ultradns.com.                  IN      TXT

;; ANSWER SECTION:
ultradns.com.           0       IN      TXT     "MS=ms21611534"
ultradns.com.           0       IN      TXT     "vvRWDjirF1kB3svP5yCIHlovQ99/rxi+VEivELNFqBvdbPZGOgtqL4qOFWfAQ0uB1o2tXEs/Ex6sgBJxaot6ig=="
ultradns.com.           0       IN      TXT     "v=spf1 exists:%{i}._i.%{d}._d.espf.agari.com include:%{d}.79.spf-protect.agari.com -all"
ultradns.com.           0       IN      TXT     "Security Issues Contact: 1-650-228-2391"
ari.beta.aridns.net.au. 0       IN      A       37.209.194.2
ari.alpha.aridns.net.au. 0      IN      A       37.209.192.2
ari.delta.aridns.net.au. 0      IN      A       37.209.198.2
ari.gamma.aridns.net.au. 0      IN      A       37.209.196.2
pdns196.ultradns.co.uk. 0       IN      A       156.154.69.196
pdns196.ultradns.com.   0       IN      A       156.154.64.196
pdns196.ultradns.org.   0       IN      A       156.154.67.196
pdns196.ultradns.info.  0       IN      A       156.154.68.196
ari.beta.aridns.net.au. 0       IN      AAAA    2001:dcd:2::2
ari.alpha.aridns.net.au. 0      IN      AAAA    2001:dcd:1::2
ari.delta.aridns.net.au. 0      IN      AAAA    2001:dcd:4::2
ari.gamma.aridns.net.au. 0      IN      AAAA    2001:dcd:3::2
pdns196.ultradns.co.uk. 0       IN      AAAA    2610:a1:1017::e8
pdns196.ultradns.biz.   0       IN      AAAA    2610:a1:1015::e8
pdns196.ultradns.com.   0       IN      AAAA    2001:502:f3ff::e8
pdns196.ultradns.net.   0       IN      AAAA    2610:a1:1014::e8
pdns196.ultradns.org.   0       IN      AAAA    2001:502:4612::e8
pdns196.ultradns.info.  0       IN      AAAA    2610:a1:1016::e8

;; Query time: 0 msec
;; SERVER: 192.168.16.1#53(192.168.16.1)
;; WHEN: Tue Aug 25 10:56:19 CEST 2020
;; MSG SIZE  rcvd: 1117

➜  ~ dig txt bing.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> txt bing.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4005
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;bing.com.                      IN      TXT

;; ANSWER SECTION:
bing.com.               0       IN      TXT     "facebook-domain-verification=09yg8uzcfnqnlqekzsbwjxyy8rdck7"
bing.com.               0       IN      TXT     "v=spf1 include:spf.protection.outlook.com -all"
bing.com.               0       IN      TXT     "v=msv1 t=6097A7EA-53F7-4028-BA76-6869CB284C54"
dns1.p09.nsone.net.     0       IN      A       198.51.44.9
dns2.p09.nsone.net.     0       IN      A       198.51.45.9
dns3.p09.nsone.net.     0       IN      A       198.51.44.73
dns4.p09.nsone.net.     0       IN      A       198.51.45.73

;; Query time: 70 msec
;; SERVER: 192.168.16.1#53(192.168.16.1)
;; WHEN: Tue Aug 25 02:12:06 CEST 2020
;; MSG SIZE  rcvd: 359

Please note that DNS server 192.168.16.1 comes from the Hyper-V Virtual Network Adapter and it is dynamically and automatically configured by WSL/ICS/Windows, so the exact DNS server's IP changes every time Windows restarts.

➜  ~ cat /etc/resolv.conf
# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry to /etc/wsl.conf:
# [network]
# generateResolvConf = false
nameserver 192.168.16.1

Here the link to the collected log and feedback item: https://aka.ms/AA9dnzo

Expected behavior

Correct DNS response like the examples below, where the ANSWER section contains only the ANSWER section and not also some info from the AUTHORITY/ADDITIONAL sections.

The following query is done using the current authoritative DNS server for ultradns.com

➜  ~ dig @pdns196.ultradns.com txt ultradns.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> @pdns196.ultradns.com txt ultradns.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61852
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 10, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ultradns.com.                  IN      TXT

;; ANSWER SECTION:
ultradns.com.           300     IN      TXT     "MS=ms21611534"
ultradns.com.           300     IN      TXT     "Security Issues Contact: 1-650-228-2391"
ultradns.com.           300     IN      TXT     "v=spf1 exists:%{i}._i.%{d}._d.espf.agari.com include:%{d}.79.spf-protect.agari.com -all"
ultradns.com.           300     IN      TXT     "vvRWDjirF1kB3svP5yCIHlovQ99/rxi+VEivELNFqBvdbPZGOgtqL4qOFWfAQ0uB1o2tXEs/Ex6sgBJxaot6ig=="

;; AUTHORITY SECTION:
ultradns.com.           3600    IN      NS      pdns196.ultradns.biz.
ultradns.com.           3600    IN      NS      pdns196.ultradns.co.uk.
ultradns.com.           3600    IN      NS      pdns196.ultradns.org.
ultradns.com.           3600    IN      NS      pdns196.ultradns.com.
ultradns.com.           3600    IN      NS      pdns196.ultradns.info.
ultradns.com.           3600    IN      NS      pdns196.ultradns.net.
ultradns.com.           3600    IN      NS      ari.beta.aridns.net.au.
ultradns.com.           3600    IN      NS      ari.gamma.aridns.net.au.
ultradns.com.           3600    IN      NS      ari.alpha.aridns.net.au.
ultradns.com.           3600    IN      NS      ari.delta.aridns.net.au.

;; Query time: 25 msec
;; SERVER: 156.154.64.196#53(156.154.64.196)
;; WHEN: Tue Aug 25 10:51:48 CEST 2020
;; MSG SIZE  rcvd: 623

The following query is done using my ISP's DNS.

➜  ~ dig @192.168.1.254 txt bing.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> @192.168.1.254 txt bing.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23185
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 8, ADDITIONAL: 17

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: c07dcc091ba95b504d06372d5f44576986e68d091921e2ec (good)
;; QUESTION SECTION:
;bing.com.                      IN      TXT

;; ANSWER SECTION:
bing.com.               3600    IN      TXT     "v=spf1 include:spf.protection.outlook.com -all"
bing.com.               3600    IN      TXT     "v=msv1 t=6097A7EA-53F7-4028-BA76-6869CB284C54"
bing.com.               3600    IN      TXT     "facebook-domain-verification=09yg8uzcfnqnlqekzsbwjxyy8rdck7"

;; AUTHORITY SECTION:
bing.com.               127543  IN      NS      dns2.p09.nsone.net.
bing.com.               127543  IN      NS      ns4-204.azure-dns.info.
bing.com.               127543  IN      NS      ns1-204.azure-dns.com.
bing.com.               127543  IN      NS      dns4.p09.nsone.net.
bing.com.               127543  IN      NS      ns2-204.azure-dns.net.
bing.com.               127543  IN      NS      dns3.p09.nsone.net.
bing.com.               127543  IN      NS      ns3-204.azure-dns.org.
bing.com.               127543  IN      NS      dns1.p09.nsone.net.

;; ADDITIONAL SECTION:
dns1.p09.nsone.net.     25148   IN      A       198.51.44.9
dns2.p09.nsone.net.     25148   IN      A       198.51.45.9
dns3.p09.nsone.net.     25160   IN      A       198.51.44.73
dns4.p09.nsone.net.     25160   IN      A       198.51.45.73
ns1-204.azure-dns.com.  343     IN      A       40.90.4.204
ns2-204.azure-dns.net.  937     IN      A       64.4.48.204
ns3-204.azure-dns.org.  2681    IN      A       13.107.24.204
ns4-204.azure-dns.info. 343     IN      A       13.107.160.204
dns1.p09.nsone.net.     25148   IN      AAAA    2620:4d:4000:6259:7::9
dns2.p09.nsone.net.     25148   IN      AAAA    2a00:edc0:6259:7::9
dns3.p09.nsone.net.     25160   IN      AAAA    2620:4d:4000:6259:7::90
dns4.p09.nsone.net.     25160   IN      AAAA    2a00:edc0:6259:7::90
ns1-204.azure-dns.com.  343     IN      AAAA    2603:1061::cc
ns2-204.azure-dns.net.  937     IN      AAAA    2620:1ec:8ec::cc
ns3-204.azure-dns.org.  2681    IN      AAAA    2a01:111:4000::cc
ns4-204.azure-dns.info. 343     IN      AAAA    2620:1ec:bda::cc

;; Query time: 23 msec
;; SERVER: 192.168.1.254#53(192.168.1.254)
;; WHEN: Tue Aug 25 02:12:26 CEST 2020
;; MSG SIZE  rcvd: 830

The following query is done using Google's public DNS server.

➜  ~ dig @8.8.8.8 txt bing.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> @8.8.8.8 txt bing.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60678
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;bing.com.                      IN      TXT

;; ANSWER SECTION:
bing.com.               3599    IN      TXT     "facebook-domain-verification=09yg8uzcfnqnlqekzsbwjxyy8rdck7"
bing.com.               3599    IN      TXT     "v=msv1 t=6097A7EA-53F7-4028-BA76-6869CB284C54"
bing.com.               3599    IN      TXT     "v=spf1 include:spf.protection.outlook.com -all"

;; Query time: 37 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Aug 25 02:15:13 CEST 2020
;; MSG SIZE  rcvd: 226

Actual behavior

➜  ~ dig txt bing.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> txt bing.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4005
;; flags: qr rd ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;bing.com.                      IN      TXT

;; ANSWER SECTION:
bing.com.               0       IN      TXT     "facebook-domain-verification=09yg8uzcfnqnlqekzsbwjxyy8rdck7"
bing.com.               0       IN      TXT     "v=spf1 include:spf.protection.outlook.com -all"
bing.com.               0       IN      TXT     "v=msv1 t=6097A7EA-53F7-4028-BA76-6869CB284C54"
dns1.p09.nsone.net.     0       IN      A       198.51.44.9
dns2.p09.nsone.net.     0       IN      A       198.51.45.9
dns3.p09.nsone.net.     0       IN      A       198.51.44.73
dns4.p09.nsone.net.     0       IN      A       198.51.45.73

;; Query time: 70 msec
;; SERVER: 192.168.16.1#53(192.168.16.1)
;; WHEN: Tue Aug 25 02:12:06 CEST 2020
;; MSG SIZE  rcvd: 359

➜  ~ dig txt ultradns.com

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> txt ultradns.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17666
;; flags: qr rd ad; QUERY: 1, ANSWER: 22, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;ultradns.com.                  IN      TXT

;; ANSWER SECTION:
ultradns.com.           0       IN      TXT     "MS=ms21611534"
ultradns.com.           0       IN      TXT     "vvRWDjirF1kB3svP5yCIHlovQ99/rxi+VEivELNFqBvdbPZGOgtqL4qOFWfAQ0uB1o2tXEs/Ex6sgBJxaot6ig=="
ultradns.com.           0       IN      TXT     "v=spf1 exists:%{i}._i.%{d}._d.espf.agari.com include:%{d}.79.spf-protect.agari.com -all"
ultradns.com.           0       IN      TXT     "Security Issues Contact: 1-650-228-2391"
ari.beta.aridns.net.au. 0       IN      A       37.209.194.2
ari.alpha.aridns.net.au. 0      IN      A       37.209.192.2
ari.delta.aridns.net.au. 0      IN      A       37.209.198.2
ari.gamma.aridns.net.au. 0      IN      A       37.209.196.2
pdns196.ultradns.co.uk. 0       IN      A       156.154.69.196
pdns196.ultradns.com.   0       IN      A       156.154.64.196
pdns196.ultradns.org.   0       IN      A       156.154.67.196
pdns196.ultradns.info.  0       IN      A       156.154.68.196
ari.beta.aridns.net.au. 0       IN      AAAA    2001:dcd:2::2
ari.alpha.aridns.net.au. 0      IN      AAAA    2001:dcd:1::2
ari.delta.aridns.net.au. 0      IN      AAAA    2001:dcd:4::2
ari.gamma.aridns.net.au. 0      IN      AAAA    2001:dcd:3::2
pdns196.ultradns.co.uk. 0       IN      AAAA    2610:a1:1017::e8
pdns196.ultradns.biz.   0       IN      AAAA    2610:a1:1015::e8
pdns196.ultradns.com.   0       IN      AAAA    2001:502:f3ff::e8
pdns196.ultradns.net.   0       IN      AAAA    2610:a1:1014::e8
pdns196.ultradns.org.   0       IN      AAAA    2001:502:4612::e8
pdns196.ultradns.info.  0       IN      AAAA    2610:a1:1016::e8

;; Query time: 0 msec
;; SERVER: 192.168.16.1#53(192.168.16.1)
;; WHEN: Tue Aug 25 10:56:19 CEST 2020
;; MSG SIZE  rcvd: 1117

Info from the AUTHORITY/ADDITIONAL sections are mixed in the ANSWER section: this behaviour currently creates issues to other programs that need to process the answer.

For example, in this issue geth cannot unmarshal the DNS message because it's greater then 512 bytes.

Geth is written in go, and go DNS client follows the RFC 1035 specification. This specification states that via UDP the maximum allowed message size is 512 bytes.

The program works fine with all other DNS servers because ANSWER configured in the DNS server is correctly less then 512 bytes, but it fails with WSL that - with the addition of other information - creates an ANSWER section too big.

This strange behavior potentially impacts every RFC 1035 compliant library, and at least it impatcs every program written in go-lang and that uses the native DNS client library.

As a final note, I don't know if it is related to the same problem or if it can provide some clues, you can also notice a warning message appearing at the beginning of the DNS response:

;; WARNING: recursion requested but not available

[therealkenc]

Thanks for the detailed report. I can't repro here, but I am pretty sure that's only because my ISP isn't coughing up any AUTHORITY/ADDITIONAL upstream. Please collect logs and backlink the feedback item. Your analysis looks awfully sound on its own merits, but having traces for the devs to look at is better than not.

[Neurone]

Sure. I updated the issue adding a query to an authoritative DNS - so you can see by your own even if your ISP does not act like mine - and I added the link to the feedback item. I also updated the new IP of the resolver (now 192.168.16.1) to reflect the recording of the feeedback.

[timriker]

This happens for me too. Any fix in the works? This gets in the way of DNS administration.

This happens for zones that are authoritative for your upstream DNS server. I see it on VPN for the office, but not when I'm not on VPN.

[florentchauveau]

This is still happening with build 10.0.19042.610.

Really surprised to see so few people affected by this.

The "ADDITIONAL SECTION" is added to the "ANSWER SECTION". This makes things really bad when you are managing thousands of targets with Ansible...

[boogie-byte]

I am affected by this issue as well.

Windows build number: 10.0.19042.685
Your Distribution version: Ubuntu 20.04
Whether the issue is on WSL 2 and/or WSL 1: WSL2 Linux version 4.19.104-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Wed Feb 19 06:37:35 UTC 2020

[timriker]

Thanks for the detailed report. I can't repro here, but I am pretty sure that's only because my ISP isn't coughing up any AUTHORITY/ADDITIONAL upstream. Please collect logs and backlink the feedback item. Your analysis looks awfully sound on its own merits, but having traces for the devs to look at is better than not.

You should be able to reproduce this by pointing your windows DNS server(s) to servers that are authoritative for something instead of to your default ISP servers or public servers like google. I don't know of any public servers off hand that are both authoritative AND allow recursive lookups. This, however, is common for corporate DNS servers when on VPN connections. Often the company DNS servers handle internal non-routable names for the company domain as well as recursing requests for external domains.