Update library does not support signatures created from bin_signing.py

[codingandmore]

Board

ESP32Dev board

Device Description

DevKit plain module

Hardware Configuration

no attachments, plan board with WiFi

Version

v3.3.7

Type

Bug

IDE Name

VSCode

Operating System

Windows 11

Flash frequency

40MHz

PSRAM enabled

yes

Upload speed

921600

Description

After applying fix from #12422 signed binaries created with the signing helper tool bin_signing.py can not be verified. The verification fails even with a correct signature and public key for RSA keys. It seems that the code for sigining uses:

        signature = private_key.sign(
            binary_data, padding.PSS(mgf=padding.MGF1(hash_obj), salt_length=padding.PSS.MAX_LENGTH), hash_obj
        )
        key_typ

Apparently the PSS algorithm with this salt length is not accepted by the mbed-tls library. It works after changing line 127 in Updater_Signing.cpp
from

 int ret = mbedtls_pk_verify((mbedtls_pk_context *)_ctx, md_type, hashBytes, hash->getHashSize(), (const unsigned char *)signature, signatureLen);

to

  // set mode and padding length according to python script for signing:
  size_t max_salt_bytes = 0;
  size_t hash_size = hash->getHashSize();
  mbedtls_rsa_context* rsa_ctx = mbedtls_pk_rsa(*(mbedtls_pk_context *)_ctx);
  if (rsa_ctx) {
    size_t key_size = mbedtls_rsa_get_len(rsa_ctx);
    max_salt_bytes = key_size - hash_size - 2;
    log_i("key size: %d, hash size: %d, salt-len: %d", key_size, hash_size, max_salt_bytes);
    if (max_salt_bytes <= 0) {
      log_e("Invalid length for salt: %d", max_salt_bytes);
      max_salt_bytes = 0;
    }
  }

  int ret;
  if (max_salt_bytes) {
    mbedtls_pk_rsassa_pss_options  options = { md_type, (int) max_salt_bytes};
    log_d("using verify ext to verify signature");
    ret = mbedtls_pk_verify_ext(MBEDTLS_PK_RSASSA_PSS, &options, (mbedtls_pk_context *) _ctx ,md_type, hashBytes, hash_size,
            (const unsigned char *)signature, signatureLen);
  } else {
    ret = mbedtls_pk_verify((mbedtls_pk_context *)_ctx, md_type, hashBytes, hash_size, (const unsigned char *)signature, signatureLen);
  }

The calculation of max_salt_bytes should be reviewed by an expert.

Another option would be to change the signing script to use parameters compatible with mbed-tls, But I could not figure out what mbded-tls uses as defaults (might also be a configuration of the library). And it seems that the current settings are the recommended ones w.r.t. security.

I could also not verify signatures generated with ECDSA keys but I did not investigate this.

Sketch

https://github.com/espressif/arduino-esp32/blob/master/libraries/Update/examples/Signed_OTA_Update/Signed_OTA_Update.ino

Debug Message

[ 52322][I][Updater.cpp:734] end(): Verifying signature: hash: 3ffd25ec, _sign: 3ffc3b1c, size: 512
[ 52331][I][Updater.cpp:736] end(): Verifying signature...
[ 52835][E][Updater_Signing.cpp:74] verify(): RSA signature verification failed: -0x4380
[ 52843][E][Updater.cpp:762] end(): Signature verification failed

Other Steps to Reproduce

The ESP32 Arduino Update Library is excellent to have. Signing OTA updates is really important and this library makes it so much easier! Thank You for making this available in the Espressif Arduino SDK.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[codingandmore]

I used RSA keys with 4096 bits and SHA512 as hash algorithm. The proposed fix may not work for all combinations.

[lucasssvaz]

When I added this example everything was working as intended. Maybe something changed on the IDF side. I will take a look once I am back from vacation.

[codingandmore]

Enjoy your vacation! To better reproduce the problem I have prepared a full sketch. I have used bin_signing.py from master branch. To prepare compilation execute the following commands:

python bin_signing.py --generate-key rsa-2048 --out priv_rsa-2048.pem
python bin_signing.py --extract-pubkey priv_rsa-2048.pem --out pub_rsa-2048.pem
python bin_signing.py --bin hello.txt --key priv_rsa-2048.pem --hash sha256 --out hello.signed
python bin_signing.py --verify hello.signed --pubkey pub_rsa-2048.pem --hash sha256
# Verifying signature of hello.signed with pub_rsa-2048.pem...
# Verifying RSA-PSS signature with SHA256
# ✓ Signature verification SUCCESSFUL!
# extract the first 256 bytes from the last 512 bytes of the file
tail -c 512 hello.signed | head -c 256 > hello.signature
xxd -i hello.signature > signature.h
xxd -i hello.txt > hello.h

The input file hello.txt is a 512 byte long text file (attached)

hello.txt

This is the sketch (it only consists of the setup() function):

#include <Arduino.h>
#include <SHA2Builder.h>
#include "mbedtls/pk.h"
#include "mbedtls/rsa.h"
#include "pub_rsa-2048.h"
#include "hello.h"
#include "signature.h"

void setup() {
  mbedtls_pk_context* ctx = new mbedtls_pk_context;
  mbedtls_pk_init((mbedtls_pk_context *)ctx);

  int ret = mbedtls_pk_parse_public_key(ctx, PUBLIC_KEY, PUBLIC_KEY_LEN);
  if (ret != 0) {
    log_e("Failed to parse RSA public key: -0x%04X", -ret);
    return;
  }

  // calculate hash, note hardcoded to sha256!
  SHA256Builder hash;
  uint8_t hash_digest[64];
  size_t hash_size;

  log_d("Calculating hash");
  hash.begin();
  hash.add(hello_txt, sizeof(hello_txt));
  hash.calculate();
  hash_size = hash.getHashSize();
  hash.getBytes(hash_digest);
  log_i("Hash Digest ,size: %d, input_size: %d", hash_size, sizeof(hello_txt));
  log_buf_i(hash_digest, hash_size);

  size_t max_salt_bytes = 0;
  mbedtls_rsa_context* rsa_ctx = mbedtls_pk_rsa(*ctx);

  if (rsa_ctx) {
    log_d("Found an RSA public key");
    size_t key_size = mbedtls_rsa_get_len(rsa_ctx);
    max_salt_bytes = key_size - hash_size - 2;
    log_d("key size: %d, hash size: %d, salt-len: %d", key_size, hash_size, max_salt_bytes);
  }

  log_d("verifying with hash size: %d, sig size: %d", hash_size, sizeof(hello_signature));
  log_d("Trying simple verify:");
  ret = mbedtls_pk_verify(ctx, MBEDTLS_MD_SHA256, hash_digest, hash_size, (const unsigned char *)hello_signature, sizeof(hello_signature));
  if (ret == 0) {
    log_i("RSA signature verified successfully");
  } else {
    log_e("RSA signature verification failed: -0x%04X", -ret);
  }

  log_d("using verify_ext to verify signature");
  mbedtls_pk_rsassa_pss_options  options = { MBEDTLS_MD_SHA256, (int) max_salt_bytes};
  ret = mbedtls_pk_verify_ext(MBEDTLS_PK_RSASSA_PSS, &options, ctx, MBEDTLS_MD_SHA256, hash_digest, hash_size,
          (const unsigned char *)hello_signature, sizeof(hello_signature));
  if (ret == 0) {
    log_i("RSA signature verified successfully");
  } else {
    log_e("RSA signature verification failed: -0x%04X", -ret);
  }
  delete ctx;
}

void loop() {
  delay(1000);
}

Executing the sketch I get this output:

============ Before Setup End ============
[   557][D][main.cpp:25] setup(): Calculating hash
[   562][I][main.cpp:31] setup(): Hash Digest ,size: 32, input_size: 512
/* 0x0000 */ 0x7a, 0xba, 0xeb, 0xc6, 0xcb, 0xc0, 0x17, 0xf8, 0xa3, 0x2c, 0xe4, 0x34, 0x3e, 0xa5, 0xdf, 0xda,    // z........,.4>...
/* 0x0010 */ 0x39, 0x5e, 0x34, 0xb1, 0xd1, 0x3e, 0x65, 0x45, 0x53, 0xf2, 0x6e, 0xe8, 0x96, 0x14, 0xe3, 0x8a,    // 9^4..>eES.n.....
[   749][D][main.cpp:38] setup(): Found an RSA public key
[   756][D][main.cpp:41] setup(): key size: 256, hash size: 32, salt-len: 222
[   765][D][main.cpp:44] setup(): verifying with hash size: 32, sig size: 256
[   774][D][main.cpp:45] setup(): Trying simple verify:
[   798][E][main.cpp:50] setup(): RSA signature verification failed: -0x4380
[   805][D][main.cpp:53] setup(): using verify_ext to verify signature
[   815][I][main.cpp:58] setup(): RSA signature verified successfully

Please also check this line:

if (!_sign->verify(_hash, _signatureBuffer, maxSigSize))

maxSigSize is always set to 512. If the actual signature is shorter verification seems to fail too.

Hope this helps.