Fix client-initiated renegotiation on OpenSSL 3.0+ (#64531)

OpenSSL 3.0 disables client-initiated renegotiation by default, which makes the server ignore all attempts at renegotiation. This makes the behavior different than with OpenSSL 1.1.1 and earlier, as well as different from windows.

This commit force-enables the client-initiated renegotiation again. All attempts at renegotiation are then handled by the managed code based on the AllowRenegotiation flag in the authentication options.

Author: rzikm

src/native/libs/System.Security.Cryptography.Native/pal_ssl.c

@@ -162,6 +162,19 @@ SSL_CTX* CryptoNative_SslCtxCreate(const SSL_METHOD* method)
         // to be to use server preference (as of June 2020), so just always assert that.
         SSL_CTX_set_options(ctx, SSL_OP_NO_COMPRESSION | SSL_OP_CIPHER_SERVER_PREFERENCE);
 
+#ifdef NEED_OPENSSL_3_0
+        if (CryptoNative_OpenSslVersionNumber() >= OPENSSL_VERSION_3_0_RTM)
+        {
+            // OpenSSL 3.0 forbids client-initiated renegotiation by default. To avoid platform
+            // differences, we explicitly enable it and handle AllowRenegotiation flag in managed
+            // code as in previous versions
+#ifndef SSL_OP_ALLOW_CLIENT_RENEGOTIATION
+#define SSL_OP_ALLOW_CLIENT_RENEGOTIATION ((uint64_t)1 << (uint64_t)8)
+#endif
+            SSL_CTX_set_options(ctx, SSL_OP_ALLOW_CLIENT_RENEGOTIATION);
+        }
+#endif
+
         // If openssl.cnf doesn't have an opinion for CipherString, then use this value instead
         if (!g_config_specified_ciphersuites)
         {