How to sign data when using pkcs11 engine for openssl with a private key stored on an HSM

[chaoshades]

Description

Hi all,

We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (libcknfast.so) provided by the vendor which is compatible.

 We also already tested the new feature added in dotnet 8 concerning the easier way to open keys from OpenSSL that works really well.

An exception is raised when we call SignData on the RSAOpenSsl instance created from the SafeEvpPKeyHandle returned by the new dotnet 8 feature OpenPrivateKeyFromEngine.

Timeline

The issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this.

We will update proactively if the status changes.

We understand full well that we have an on-premise HSM that is not easily accessible for you to test.
We will make us available if there is a need to interact or experiment things.

Reproduction Steps

Using this snippet of code :

byte[] data = ...;
using SafeEvpPKeyHandle privateKeyHandle = SafeEvpPKeyHandle.OpenPrivateKeyFromEngine("pkcs11", "pkcs11:type=private;object=my-private-key");
using RSAOpenSsl rsaPri = new(privateKeyHandle);
rsaPri.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);

Expected behavior

The data is signed using the private key stored into the HSM.

Actual behavior

rsaPri.SignData(...) raises the following exception :

Interop+Crypto+OpenSslCryptographicException: error:04084093:rsa routines::value missing
at Interop.Crypto.RsaSignHash(SafeEvpPKeyHandle pkey, RSASignaturePaddingMode paddingMode, IntPtr digestAlgorithm, ReadOnlySpan`1 hash, Span`1 destination)
at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan`1 hash, Span`1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature)
at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)

Configuration

OS

sh-4.4# cat /etc/os-release

NAME="Red Hat Enterprise Linux"
VERSION="8.8 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.8
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"

Dotnet

sh-4.4# dotnet --info

Host:
  Version:      8.0.0-rc.2.23470.7
  Architecture: x64
  Commit:       49bf70a429
  RID:          linux-x64

.NET SDKs installed:
  No SDKs were found.

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.0-rtm.23471.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.0-rc.2.23470.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

OpenSSL

sh-4.4# openssl version

OpenSSL 1.1.1k  FIPS 25 Mar 2021

sh-4.4# openssl engine pkcs11 -t -v

(pkcs11) pkcs11 engine
     [ available ]
     SO_PATH, MODULE_PATH, PIN, VERBOSE, QUIET, INIT_ARGS, FORCE_LOGIN

sh-4.4# cat /etc/pki/tls/openssl.cnf

openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib64/engines-1.1/pkcs11.so
MODULE_PATH = /opt/nfast/toolkits/pkcs11/libcknfast.so
init = 0  

Key

Private Key Pkcs11 Extensions

p11tool --provider /opt/nfast/toolkits/pkcs11/libcknfast.so --list-privkeys

Object 0:
        URL: pkcs11:...object=my-private-key;type=private
        Type: Private key (RSA-2048)
        Label: my-private-key
        Flags: CKA_WRAP/UNWRAP; CKA_SENSITIVE; 
        ID: ...

Other information

Following the above stack trace, it brings us here :
https://github.com/dotnet/runtime/blob/df8cbd11b84a5930ec71666ad87479d559872718/src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c#L254C1-L263C6

We did some trial and error to determine where the root cause was :

1. We tried using purely OpenSSL commands.
2. We tried using C code, that was provided to us by @krwq (greatly appreciated 🥇 )

   We made contact through our Microsoft account manager

Please note that we cut the pkcs11 uri in the following samples for brevity.

The ones we are using were provided by tools like p11tool or from the HSM-vendor tool.

Using OpenSSL

Output

echo -n "Hello, world!" > /tmp/test.txt

(No output)

openssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=my-private-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txt

engine "pkcs11" set.

openssl dgst -engine pkcs11 -verify "pkcs11:type=public;object=my-public-key" -sha256 -keyform engine -signature /tmp/test.sig -binary /tmp/test.txt

engine "pkcs11" set.
Verified OK

Everything went well.
So we believe this would eliminate potential issues that could be on our end with OpenSSL configurations and that the proprietary module is not fiddling with the process.

Using C code

Output

ossl_test.c.txt
build.sh.txt

sh-4.4# ./build.sh

Compilation successful. Executable is named ossl_test.

sh-4.4# ./ossl_test

Main - Got Private Key!
Got RSA!
RSA_FLAG_EXT_PKEY:32
d:1
Factors - p:1 q:1
CRT Params - dmp1:1 dmq1:1 iqmp:1
Main - HasNoPrivateKey Result:1
Signature verification succeeded.
Main - Got Signature!

Let's explain the output a little :

• Main - Got Private Key! means that the pkcs11 engine was initialized and the private key was loaded accordingly.
• Got RSA! until Main - HasNoPrivateKey... are printf used to debug the HasNoPrivateKey function.

  The value next to each key property (d, p, q, etc) are boolean that checks if it is NULL.
  In that case, everything is expected to be NULL because the key is not exportable from the HSM.

• Main - Got Signature! means the signature was done correctly using the pkcs11 engine.

Everything went well for the signature.

However, HasNoPrivateKey returns 1.
Which means it looks like there are no private key and then dotnet will raise the exception above : error:04084093:rsa routines::value missing

We added the RSA_FLAG_EXT_PKEY flag in the output because we found an older issue that looked like ours with a different context.

We need some guidance if there is something that could be done about the HasNoPrivateKey function or if there something to do with the RSA_FLAG_EXT_PKEY flag.

We created the issue as requested by our Microsoft account manager to allow you and/or the community to have a handle on something more tangible.

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Hi all,

We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (libcknfast.so) provided by the vendor which is compatible.

 We also already tested the new feature added in dotnet 8 concerning the easier way to open keys from OpenSSL that works really well.

An exception is raised when we call SignData on the RSAOpenSsl instance created from the SafeEvpPKeyHandle returned by the new dotnet 8 feature OpenPrivateKeyFromEngine.

Timeline

The issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this.

We will update proactively if the status changes.

We understand full well that we have an on-premise HSM that is not easily accessible for you to test.
We will make us available if there is a need to interact or experiment things.

Reproduction Steps

Using this snippet of code :

byte[] data = ...;
using SafeEvpPKeyHandle privateKeyHandle = SafeEvpPKeyHandle.OpenPrivateKeyFromEngine("pkcs11", "pkcs11:type=private;object=my-private-key");
using RSAOpenSsl rsaPri = new(privateKeyHandle);
rsaPri.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);

Expected behavior

The data is signed using the private key stored into the HSM.

Actual behavior

rsaPri.SignData(...) raises the following exception :

Interop+Crypto+OpenSslCryptographicException: error:04084093:rsa routines::value missing
at Interop.Crypto.RsaSignHash(SafeEvpPKeyHandle pkey, RSASignaturePaddingMode paddingMode, IntPtr digestAlgorithm, ReadOnlySpan`1 hash, Span`1 destination)
at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan`1 hash, Span`1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature)
at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)

Configuration

OS

sh-4.4# cat /etc/os-release

NAME="Red Hat Enterprise Linux"
VERSION="8.8 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.8
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"

Dotnet

sh-4.4# dotnet --info

Host:
  Version:      8.0.0-rc.2.23470.7
  Architecture: x64
  Commit:       49bf70a429
  RID:          linux-x64

.NET SDKs installed:
  No SDKs were found.

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.0-rtm.23471.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.0-rc.2.23470.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

OpenSSL

sh-4.4# openssl version

OpenSSL 1.1.1k  FIPS 25 Mar 2021

sh-4.4# openssl engine pkcs11 -t -v

(pkcs11) pkcs11 engine
     [ available ]
     SO_PATH, MODULE_PATH, PIN, VERBOSE, QUIET, INIT_ARGS, FORCE_LOGIN

sh-4.4# cat /etc/pki/tls/openssl.cnf

openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib64/engines-1.1/pkcs11.so
MODULE_PATH = /opt/nfast/toolkits/pkcs11/libcknfast.so
init = 0  

Key

Private Key Pkcs11 Extensions

p11tool --provider /opt/nfast/toolkits/pkcs11/libcknfast.so --list-privkeys

Object 0:
        URL: pkcs11:...object=my-private-key;type=private
        Type: Private key (RSA-2048)
        Label: my-private-key
        Flags: CKA_WRAP/UNWRAP; CKA_SENSITIVE; 
        ID: ...

Other information

Following the above stack trace, it brings us here :
https://github.com/dotnet/runtime/blob/df8cbd11b84a5930ec71666ad87479d559872718/src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c#L254C1-L263C6

We did some trial and error to determine where the root cause was :

1. We tried using purely OpenSSL commands.
2. We tried using C code, that was provided to us by @krwq (greatly appreciated 🥇 )

   We made contact through our Microsoft account manager

Please note that we cut the pkcs11 uri in the following samples for brevity.

The ones we are using were provided by tools like p11tool or from the HSM-vendor tool.

Using OpenSSL

Output

echo -n "Hello, world!" > /tmp/test.txt

(No output)

openssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=my-private-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txt

engine "pkcs11" set.

openssl dgst -engine pkcs11 -verify "pkcs11:type=public;object=my-public-key" -sha256 -keyform engine -signature /tmp/test.sig -binary /tmp/test.txt

engine "pkcs11" set.
Verified OK

Everything went well.
So we believe this would eliminate potential issues that could be on our end with OpenSSL configurations and that the proprietary module is not fiddling with the process.

Using C code

Output

ossl_test.c.txt
build.sh.txt

sh-4.4# ./build.sh

Compilation successful. Executable is named ossl_test.

sh-4.4# ./ossl_test

Main - Got Private Key!
Got RSA!
RSA_FLAG_EXT_PKEY:32
d:1
Factors - p:1 q:1
CRT Params - dmp1:1 dmq1:1 iqmp:1
Main - HasNoPrivateKey Result:1
Signature verification succeeded.
Main - Got Signature!

Let's explain the output a little :

• Main - Got Private Key! means that the pkcs11 engine was initialized and the private key was loaded accordingly.
• Got RSA! until Main - HasNoPrivateKey... are printf used to debug the HasNoPrivateKey function.

  The value next to each key property (d, p, q, etc) are boolean that checks if it is NULL.
  In that case, everything is expected to be NULL because the key is not exportable from the HSM.

• Main - Got Signature! means the signature was done correctly using the pkcs11 engine.

Everything went well for the signature.

However, HasNoPrivateKey returns 1.
Which means it looks like there are no private key and then dotnet will raise the exception above : error:04084093:rsa routines::value missing

We added the RSA_FLAG_EXT_PKEY flag in the output because we found an older issue that looked like ours with a different context.

We need some guidance if there is something that could be done about the HasNoPrivateKey function or if there something to do with the RSA_FLAG_EXT_PKEY flag.

We created the issue as requested by our Microsoft account manager to allow you and/or the community to have a handle on something more tangible.

Author:	chaoshades
Assignees:	-
Labels:	area-System.Security
Milestone:	-

[krwq]

@chaoshades I've played around with SoftHSM to generate keys for OpenSC's pkcs11 engine and tpm2tss with tpm keys and I've convinced myself this is specific to your pkcs11 engine implementation and not strictly a bug in .NET.
As a side note tpm2tss has some issues with RSA keys and it don't seem to work in that combination (RSA doesn't seem to be implemented yet).

I believe your pkcs11 implementation should be setting following flag on the RSA key when creating EVP_PKEY instance:

//RSA* rsa = ... (wherever you create it in your pkcs11 code)
RSA_set_flags(rsa, RSA_FLAG_EXT_PKEY);

if some older OpenSSL then:
rsa->flags |= RSA_FLAG_EXT_PKEY;

having said that the check we do seems redundant and it seems to be causing issue with at least one implementation - while it provides slightly better exception message when you accidentally pass public key for something expecting private key I'm not sure extra work and extra issue it causes is worth it (and we should probably let OpenSSL decide what's best error to display) so I'll still consider removing HasNoPrivateKey and fixing this issue in .NET.

[chaoshades]

@krwq We will ask our vendor if they set this flag in their module.

About the HasNoPrivateKey, if I remember correctly, this check protect from segfault. So maybe removing it would be easier to do by moving to OpenSSL 3.0 since this will be handled entirely by OpenSSL then.

[jeffhandley]

Based on the findings above and the anticipation that this issue can be addressed without a change to .NET 8, I'm going to move this issue to the 9.0.0 milestone so that it no longer shows up as blocking the 8.0.0 release.

If the end-to-end does still end up being blocked and we find a change is necessary into .NET 8 for the scenario to fully work, we can explore a fix into .NET 9 first with the potential to backport into .NET 8 servicing. Thanks @krwq and @chaoshades for the close collaboration here!

[chaoshades]

We got feedback from the vendor and the team managing the HSM and the actual pkcs11 engine implementation doesn't set the RSA_FLAG_EXT_PKEY flag. Since this flag is also deprecated in favor of OpenSSL 3.0, we kinda understand adding deprecated new things in the engine implementation would be harder.

We also believes a workaround could be possible, like a native call or something that simulate RSA_set_flags(rsa, RSA_FLAG_EXT_PKEY); that would be called before SignData to let the HasNoPrivateKey function react with the expected behavior.

However, our client doesn't want to maintain a fix over .NET 8 for production use. The expertise is quite niche and they prefer to wait rather than risk it. So, moving this issue as non-blocking the .NET 8 release is fine for us. We will continue to follow the issue because as we said above : We will make us available if there is a need to interact or experiment things.

I may experiment on the workaround in my free time, just to see if my previous idea could work.

[jeffhandley]

@chaoshades We implemented OpenSSL Providers support in .NET 9. Have you moved to OpenSSL 3.x by chance? It's unlikely that we'll add new OpenSSL 1.x ENGINE features.