BearSSL takes more than twice as long for handshake as axTLS

[marcelstoer]

This is a follow-up to #4738 (comment)

Claim

I observed that BearSSL in standard configuration takes more than twice as long for a handshake as axTLS.

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: WeMos D1 Mini Pro
• Core Version: 2.4.2
• Development Env: Arduino IDE
• Operating System: macOS

Sample sketch

The below sketch is inspired by what @artua presented at #4738 but it's much simpler. I feel that when just starting out measuring / comparing performance you shouldn't collect too much data at once as you might not see the wood for the trees. Once you identify a suspect you then adjust the rig accordingly and collect more data around the suspected culprit.

The program establishes a connection to a SSL/TLS protected host 50 times with a delay of 1.5s. For every attempt it reports total failure/success count, accumulated handshake time, average handshake time and free heap at the end. No server certificates or fingerprints are verified!

Around line 40 you define whether to use BearSSL (with setInsecure()) or axTLS.

#include <ESP8266WiFi.h>

#define HOST "thingpulse.com"
#define PORT 443
#define WIFI_SSID "ssid"
#define WIFI_PWD "password"

const char* ssid = WIFI_SSID;
const char* password = WIFI_PWD;

int numberOfSamples = 50;
int totalCounter = 0;
int failCounter = 0;
uint64_t totalDuration = 0;
void setup () {
  Serial.begin(115200);
  Serial.println("");
  WiFi.begin(ssid, password);

  Serial.print("Connecting to Wifi");
  while (WiFi.status() != WL_CONNECTED) {
    delay(300);
    Serial.print(".");
  }
  Serial.println("done.");
}


void loop() {
  if (totalCounter < numberOfSamples) {
    bool success = connectToHost();
    delay(1500);
  }
}

bool connectToHost() {
  uint64_t startTime = millis();
//  BearSSL::WiFiClientSecure client;
//  client.setInsecure();
  WiFiClientSecure client;

  boolean success = false;
  totalCounter++;
  if (client.connect(HOST, PORT)) {
    // Serial.printf("Connection to %s success.\n", HOST);
    success = true;
  } else {
    failCounter++;
    // Serial.printf("Connection to %s failed.\n", HOST);
  }
  client.flush(); // not actually needed
  client.stop();
  totalDuration += millis() - startTime;
  Serial.printf("Failed: %d, Total: %d (%f %%), Total Dur: %d, Avg. Dur: %f, Heap: %d\n", 
    failCounter, 
    totalCounter, 
    (100.0 * failCounter / totalCounter), 
    totalDuration,
    1.0 * totalDuration / totalCounter, 
    ESP.getFreeHeap());
  return success;
}

Test results

I ran the above sketch with all combinations of BearSSL/axTLS, lwIP v2 lower memory/higher bandwidth and 80/160MHz on core 2.4.2. So, this makes 2^3 * 50 samples.

The summary is as follows:
0 connection failures. That's remarkable as this whole endeavour started out with this issue https://stackoverflow.com/q/52143894.

BearSSL

• lwIP v2 lower memory, 80 MHz: ~860ms
• lwIP v2 higher bandwidth, 80 MHz: ~845ms
• lwIP v2 lower memory, 160 MHz: ~510ms
• lwIP v2 higher bandwidth, 160 MHz: ~500ms

axTLS

• lwIP v2 lower memory, 80 MHz: ~360ms
• lwIP v2 higher bandwidth, 80 MHz: ~350ms
• lwIP v2 lower memory, 160 MHz: ~250ms
• lwIP v2 higher bandwidth, 160 MHz: ~240ms

-> BearSSL takes more than twice as long
-> As expected higher bandwidth has close to zero impact as only a socket connection with handshake was established. No resources were loaded from the host.

The detailed result is in the below PDF.
BearSSL-vs-axTLS.pdf

Over at #4738 @earlephilhower stated that

BearSSL often negotiates a more secure but slower running cypher... limiting the receiving end's connection capabilities in your SSL configuration

I would first have to look into that. Not familiar with SSL config yet.

[earlephilhower]

@marcelstoer we can change your local BearSSL to only use the ciphermodes supported by axtls. In libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp there's a structure, suites_P.

Replace the long list with just the following to match the very limited scope of axTLS:

BR_TLS_RSA_WITH_AES_256_CBC_SHA256,
BR_TLS_RSA_WITH_AES_128_CBC_SHA256,
BR_TLS_RSA_WITH_AES_256_CBC_SHA,
BR_TLS_RSA_WITH_AES_128_CBC_SHA

And you can re-compile and run on your same configuration and hardware to get the numbers, w/o any changes to your own test code.

[devyte]

CC @llongeri

[marcelstoer]

@earlephilhower thanks for the valuable information. I tested what you proposed and the effect is quite striking:

lwIP v2 lower memory, 80 MHz: ~420ms

-> still 15% more than with axTLS
-> less than half of what you get with the BearSSL default config

lwIP v2 lower memory, 160 MHz: ~310ms

I just wish we could configure such fundamentals in the sketch rather than tinkering with the core's .cpp files (possible adjusting for every project).

CC @squix78

[earlephilhower]

@marcelstoer, appreciate the concrete testing numbers!

If you're power constrained than I suggest you look at 160MHz operation. My gut tells me you're not going to use much more current than 80MHz, but will need to run for ~40% less time so it will have lower total power. But that's orthogonal to BearSSL/axTLS.

Adding a call to allow you to select the cipher list (or pass in a list) is not a big deal, and I think it would cover you here. The more secure ones can end up taking more CPU, so it's a trade off here where the BearSSL chosen default is security over performance.

Another option would be to support multiple configurations in the GUI, like w/LWIP. For example, I'm compiling with a setting which minimizes RAM usage (I think 1-2KB RAM saved, so it's not trivial) but causes, potentially, slower operation in the main SSL loop. Turning off that switch would trade RAM for performance.

[marcelstoer]

Another option would be to support multiple configurations in the GUI

I would argue that this configuration belongs to the application and should be store there. The cipher suites you select are dependent on the needs of the application and the remote resources it talks to.

[earlephilhower]

Agree with you, I think I was unclear. The WiFiClientSecure::setCipher() call should be per-app and available any time.

I was suggesting providing a "BearSSL Low Memory" and a "BearSSL High Performance" version of the library that's compiled differently. Like LWIP is done today, it's a link-time option. The "High Performance" will have the same codepaths as the low-mem, but use different GCC switches which will reduce time spent doing some of the SSL operations (but use RAM).

[earlephilhower]

@marcelstoer since you've got a good testing infrastructure already, could you unzip the file below and replace the libbearssl.a in tools/sdk/lib with it and give your minimal-cipher test case another try? I'm trying to see if it makes sense to include the version that trades RAM for (hopefully) speed...

esp8266-fast.zip

[artua]

@earlephilhower <https://github.com/earlephilhower> right. I thought it would be twice power consumption at 160Mhz operation but tested power consumption is nearly the same as 80Mhz, so it is the way we can reduce ssl handshake time dramatically.

…

On Mon, Sep 17, 2018 at 6:49 PM Earle F. Philhower, III < ***@***.***> wrote: @marcelstoer <https://github.com/marcelstoer>, appreciate the concrete testing numbers! If you're power constrained than I suggest you look at 160MHz operation. My gut tells me you're not going to use much more current than 80MHz, but will need to run for ~40% less time so it will have lower total power. But that's orthogonal to BearSSL/axTLS. Adding a call to allow you to select the cipher list (or pass in a list) is not a big deal, and I think it would cover you here. The more secure ones can end up taking more CPU, so it's a trade off here where the BearSSL chosen default is security over performance. Another option would be to support multiple configurations in the GUI, like w/LWIP. For example, I'm compiling with a setting which minimizes RAM usage (I think 1-2KB RAM saved, so it's not trivial) but causes, potentially, slower operation in the main SSL loop. Turning off that switch would trade RAM for performance. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5110 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGV_3Y4Hhi_vhwhAXoGK_u82X1iMKqSBks5ub8SEgaJpZM4Wbqae> .

[marcelstoer]

@earlephilhower will do

160Mhz operation but tested power consumption is nearly the same as 80Mhz, so it is the way
we can reduce ssl handshake time dramatically

Did you time this? My test results (see above) show that CPU frequency has only negligible impact on the actual handshake time.

[artua]

With default BearSSL configuration it has great impact on timings. I will find the numbers and post them here. I think that was your numbers in another thread: 1. axTLS: median 368ms, average 390ms 2. BearSSL 80MHz: 888ms, 891ms -> over 500ms increase (140% extra) 3. BearSSL 160MHz: 568ms, 538ms -> 50% extra

…

On Tue, Sep 18, 2018 at 9:59 AM Marcel Stör ***@***.***> wrote: @earlephilhower <https://github.com/earlephilhower> will do 160Mhz operation but tested power consumption is nearly the same as 80Mhz, so it is the way we can reduce ssl handshake time dramatically Did you time this? *My* test results (see above) show that CPU frequency has only negligible impact on the actual *handshake* time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5110 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGV_3SI59i9jQ-UteVbyCNy6EwYWTV8yks5ucJmygaJpZM4Wbqae> .

[earlephilhower]

@marcelstoer For CPU-bound stuff, it should increase performance (up to the point when the slow flash interface limits it). @artua's numbers (maybe yours?) are actually comparing EC vs. RSA handshakes, so they're not a good comparison between axtls and BearSSL. However, the (2) BearSSL and (3) BearSSL numbers can be compared (they're both EC handshakes) and it's a significant speedup. RSA handshake, I don't know yet, it could be RTT limited over the network.

If you get a few minutes, I'd appreciate testing the ZIPped library from a couple back with your RSA-hacked BearSSL core. That's got a different compile option which may speed it up...

[artua]

Why I’m switching to BearSSL is the modern ciphers used by Cloudflare. axTLS didn’t support that. I need to use their cdn proxies for my application, so BearSSL is the only way to connect... Their current configuration is: ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; ssl_ecdh_curve X25519:P-256:P-384:P-224:P-521; ssl_ciphers '[ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-RSA-AES128-GCM-SHA256|ECDHE-RSA-CHACHA20-POLY1305]:ECDHE+AES128:RSA+AES128:ECDHE+AES256:RSA+AES256:ECDHE+3DES:RSA+3DES'; ssl_prefer_server_ciphers on; I think we should try chacha20 or poly for the best results?

…

On Tue, Sep 18, 2018 at 6:37 PM Earle F. Philhower, III < ***@***.***> wrote: @marcelstoer <https://github.com/marcelstoer> For CPU-bound stuff, it should increase performance (up to the point when the slow flash interface limits it). @artua <https://github.com/artua>'s numbers (maybe yours?) are actually comparing EC vs. RSA handshakes, so they're not a good comparison between axtls and BearSSL. However, the (2) BearSSL and (3) BearSSL numbers can be compared (they're both EC handshakes) and it's a significant speedup. RSA handshake, I don't know yet, it could be RTT limited over the network. If you get a few minutes, I'd appreciate testing the ZIPped library from a couple back with your RSA-hacked BearSSL core. That's got a different compile option which may speed it up... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5110 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGV_3TpAiE0SiXq4nqj8lFX4Ulxwma00ks5ucRMngaJpZM4Wbqae> .

[marcelstoer]

@earlephilhower The verdict is in...

In #5110 (comment) I added the 160MHz numbers for the original "minimal ciphers" test case. I would say that ~300ms is satisfactorily low.

Then I tried your new libbearssl.a with both 80MHz and 160MHz. The speed improvement is less than 5%. The extra RAM is negligible.

What I observed in many test runs are two "anomalies":

• The first sample is usually the fastest, then the average handshake time keeps increasing up to around sample 30 before it stablizes. One explanation is of course simple math as small spikes become less significant when the number of samples increases.
• Often somewhere between sample 5 and sample 10 there is 1 - always 1 - that takes ~3s (i.e. 10x more than the previous and the following sample).

Conclusion for my battery powered project

• use BearSSL
• slash the number of cipher suites i.e. remove the costly ones
• run at 160MHz

[earlephilhower]

Thanks for the update, @marcelstoer . Given that, I don't see any reason to offer two versions of the BearSSL library (mem vs speed). I still see the need for a "use these ciphers" and an axtls_ciphers constant default. That's a pretty trivial addition and will let you use faster, less secure ones while @artua can then choose only to use the newer (possibly slower and more secure) ones.

[earlephilhower]

@artua , PR #5151 adds a call where you can specify a single BearSSL-format cipher. You could try hardcoding the ChaCha20 using it. I have no idea what the performance difference would be, though.

[devyte]

@marcelstoer #5151 was merged, which autoclosed this. In the interest of moving forward, I'm leaving it closed for now. If you think further changes are needed, please provide details in a new issue.
Thanks again for the numbers!

[earlephilhower]

@marcelstoer , check out my latest PR, #5160. Using SSL sessions makes reconnection times 80% faster or so!