RemoteCertificateValidationCallback can not detect some incomplete certificate chains

[stephenweaver]

Description

It appears that RemoteCertificateValidationCallback (tested with SslStream and HttpWebRequest ) incorrectly validates some certificates with chain issues (ex. https://incomplete-chain.badssl.com). The code I'm attempting to write validates certificates, similar to what this tool is doing https://www.sslshopper.com/ssl-checker.html.

There does not appear to be a way to detect these issues, which is why this feels like a bug to me.

Here is my attempt at overriding the default behavior, but by the time my code it hit, all 3 certificates look like they exist. (https://dotnetfiddle.net/2aybuI)

using System;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;

public class Program
{
	public static void Main(string[] args)
	{
		var request = (HttpWebRequest)WebRequest.Create("https://incomplete-chain.badssl.com");
		request.AllowAutoRedirect = false;
		request.ServerCertificateValidationCallback = ServerCertificateValidationCallback;
		var response = (HttpWebResponse)request.GetResponse();
		response.Close();
	}

	private static bool ServerCertificateValidationCallback(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
	{
		if (sslPolicyErrors != SslPolicyErrors.None)
		{
			return false;
		}

		var newChain = new X509Chain();
		newChain.ChainPolicy.DisableCertificateDownloads = true;
		
		// Skip the leaf cert and stop short of the root cert.      
		X509ChainElementCollection chainElements = chain.ChainElements;
		for (int i = 1; i < chainElements.Count - 1; i++)
		{
			newChain.ChainPolicy.ExtraStore.Add(chainElements[i].Certificate);
		}
		var result = newChain.Build(chainElements[0].Certificate);
		
		// This is True and I want it to be False
		Console.WriteLine($"This is {result} and I want it to be False");
		
		return result;
	}
}

Configuration

Here is a sample app using .NET 5 (my code base is .NET core 3.1 with the same issue)

Here's a .NET Fiddle demonstrating the code above.
https://dotnetfiddle.net/2aybuI

Regression?

Tested on several versions of .NET and this bug has been around for a long time.

Other information

curl requests, for example, fail how I want my requests to fail if the cert chain is not valid.

 curl get https://incomplete-chain.badssl.com
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (6) Could not resolve host: get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

openssl also correctly shows these incomplete-chains as invalid

$ openssl s_client -showcerts -connect incomplete-chain.badssl.com:443
CONNECTED(00000188)
---
Certificate chain
 0 s:C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com
   i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
-----BEGIN CERTIFICATE-----
the certificate 
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com

issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2414 bytes and written 455 bytes
Verification error: unable to verify the first certificate
---

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

It appears that RemoteCertificateValidationCallback (tested with SslStream and HttpWebRequest ) incorrectly validates some certificates with chain issues (ex. https://incomplete-chain.badssl.com). The code I'm attempting to write validates certificates, similar to what this tool is doing https://www.sslshopper.com/ssl-checker.html.

There does not appear to be a way to detect these issues, which is why this feels like a bug to me.

Here is my attempt at overriding the default behavior, but by the time my code it hit, all 3 certificates look like they exist. (https://dotnetfiddle.net/2aybuI)

using System;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;

public class Program
{
	public static void Main(string[] args)
	{
		var request = (HttpWebRequest)WebRequest.Create("https://incomplete-chain.badssl.com");
		request.AllowAutoRedirect = false;
		request.ServerCertificateValidationCallback = ServerCertificateValidationCallback;
		var response = (HttpWebResponse)request.GetResponse();
		response.Close();
	}

	private static bool ServerCertificateValidationCallback(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
	{
		if (sslPolicyErrors != SslPolicyErrors.None)
		{
			return false;
		}

		var newChain = new X509Chain();
		newChain.ChainPolicy.DisableCertificateDownloads = true;
		
		// Skip the leaf cert and stop short of the root cert.      
		X509ChainElementCollection chainElements = chain.ChainElements;
		for (int i = 1; i < chainElements.Count - 1; i++)
		{
			newChain.ChainPolicy.ExtraStore.Add(chainElements[i].Certificate);
		}
		var result = newChain.Build(chainElements[0].Certificate);
		
		// This is True and I want it to be False
		Console.WriteLine($"This is {result} and I want it to be False");
		
		return result;
	}
}

Configuration

Here is a sample app using .NET 5 (my code base is .NET core 3.1 with the same issue)

Here's a .NET Fiddle demonstrating the code above.
https://dotnetfiddle.net/2aybuI

Regression?

Tested on several versions of .NET and this bug has been around for a long time.

Other information

curl requests, for example, fail how I want my requests to fail if the cert chain is not valid.

 curl get https://incomplete-chain.badssl.com
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (6) Could not resolve host: get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

openssl also correctly shows these incomplete-chains as invalid

$ openssl s_client -showcerts -connect incomplete-chain.badssl.com:443
CONNECTED(00000188)
---
Certificate chain
 0 s:C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com
   i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
-----BEGIN CERTIFICATE-----
the certificate 
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com

issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2414 bytes and written 455 bytes
Verification error: unable to verify the first certificate
---

Author:	stephenweaver
Assignees:	-
Labels:	area-System.Security, untriaged
Milestone:	-

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

It appears that RemoteCertificateValidationCallback (tested with SslStream and HttpWebRequest ) incorrectly validates some certificates with chain issues (ex. https://incomplete-chain.badssl.com). The code I'm attempting to write validates certificates, similar to what this tool is doing https://www.sslshopper.com/ssl-checker.html.

There does not appear to be a way to detect these issues, which is why this feels like a bug to me.

Here is my attempt at overriding the default behavior, but by the time my code it hit, all 3 certificates look like they exist. (https://dotnetfiddle.net/2aybuI)

using System;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;

public class Program
{
	public static void Main(string[] args)
	{
		var request = (HttpWebRequest)WebRequest.Create("https://incomplete-chain.badssl.com");
		request.AllowAutoRedirect = false;
		request.ServerCertificateValidationCallback = ServerCertificateValidationCallback;
		var response = (HttpWebResponse)request.GetResponse();
		response.Close();
	}

	private static bool ServerCertificateValidationCallback(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
	{
		if (sslPolicyErrors != SslPolicyErrors.None)
		{
			return false;
		}

		var newChain = new X509Chain();
		newChain.ChainPolicy.DisableCertificateDownloads = true;
		
		// Skip the leaf cert and stop short of the root cert.      
		X509ChainElementCollection chainElements = chain.ChainElements;
		for (int i = 1; i < chainElements.Count - 1; i++)
		{
			newChain.ChainPolicy.ExtraStore.Add(chainElements[i].Certificate);
		}
		var result = newChain.Build(chainElements[0].Certificate);
		
		// This is True and I want it to be False
		Console.WriteLine($"This is {result} and I want it to be False");
		
		return result;
	}
}

Configuration

Here is a sample app using .NET 5 (my code base is .NET core 3.1 with the same issue)

Here's a .NET Fiddle demonstrating the code above.
https://dotnetfiddle.net/2aybuI

Regression?

Tested on several versions of .NET and this bug has been around for a long time.

Other information

curl requests, for example, fail how I want my requests to fail if the cert chain is not valid.

 curl get https://incomplete-chain.badssl.com
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (6) Could not resolve host: get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

openssl also correctly shows these incomplete-chains as invalid

$ openssl s_client -showcerts -connect incomplete-chain.badssl.com:443
CONNECTED(00000188)
---
Certificate chain
 0 s:C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com
   i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
-----BEGIN CERTIFICATE-----
the certificate 
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = California, L = Walnut Creek, O = Lucas Garron Torres, CN = *.badssl.com

issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2414 bytes and written 455 bytes
Verification error: unable to verify the first certificate
---

Author:	stephenweaver
Assignees:	-
Labels:	area-System.Net.Security, untriaged
Milestone:	-

[bartonjs]

Tangent: Not specifically the problem you're talking about, but

if (sslPolicyErrors != SslPolicyErrors.None)
{
    return false;
}

means that if the system hadn't already filled in the holes you're not going to look at it.

[bartonjs]

.NET's certificate chains, on Windows, are built via Windows CAPI's CertGetCertificateChain . The CAPI chain engine has a lot of places that it looks for filling in holes:

• The collection that we provide from chain.ChainPolicy.ExtraStore
• The CurrentUser\CA (intermediate CAs) store
• The LocalMachine\CA store
• The CurrentUser\Root (trusted root CAs) store
• The LocalMachine\Root store
• The CurrentUser\ThirdPartyRoot store
  • NOT the LocalMachine\ThirdPartRoot store... for... reasons.
• The CurrentUser\My ("Personal") store
• The LocalMachine\My store
• HTTP or LDAP based caIssuers URIs from the certificate's Authority Information Access extension
  • This can be disabled, CERT_CHAIN_DISABLE_AIA, which we call cert.ChainPolicy.DisableCertificateDownloads
• Maybe a "things I've recently downloaded" cache
• Maybe other caches

Of those, only one can be turned off.

That means that you can't really use X509Chain alone to determine this scenario. What you really want to know is "did the contents from the handshake look like the chain, except that it was missing the root?". And that means that you'd need to see what was actually put on the wire.

It might be true that chain.ChainPolicy.ExtraStore in the callback already matches what was on the handshake (order and content)... but it might also (instead) be the case that SChannel already built the chain once, filled in any missing pieces, and then we already got the normalized chain. I'm not sure. If it's the normalized chain already (and maybe "anyways"), then maybe we have a need for API exposing what was on the wire.

[stephenweaver]

Hey @bartonjs , you're absolutely correct. Do you know of a way to see what was actually put on the wire? Everything I can find in the docs abstracts this away from the developer.

I don't really care if the system I'm running on is smart enough to figure out if the request safe, I need to figure out if every system/language/platform is also going come to the same conclusion. (The curl and openssl request are simple examples of this)

Yep, this is just the default behavior, and I'm cool with returning early here.

if (sslPolicyErrors != SslPolicyErrors.None)
{
    return false;
}

[bartonjs]

The one thing to investigate is if chain.ChainPolicy.ExtraStore looks like what went on the wire (after the end-entity/leaf cert). e.g. for the incomplete-chain.badssl.com test it would be empty. If so, then huzzah. If not, then there's probably no current way to get the raw form.

[stephenweaver]

Yeah it looks like if this is the only way to access the cert info, then this currently isn't possible.