feat(nimble): Replace Bluedroid with NimBLE except for ESP32

[lucasssvaz]

Description

This pull request updates Bluetooth configurations across multiple defconfig files for various ESP32 chipsets to standardize the use of the NimBLE stack and configure Bluetooth to operate in BLE-only mode. The most important changes involve enabling NimBLE while disabling the Bluedroid stack and setting specific Bluetooth controller modes.

[cpatulea]

@lucasssvaz I'm working with the Meshtastic project to port to Arduino v3.x. We support many different devices, including original ESP32, ESP32-S3, etc. ESP32-S3 with NimBLE works. But ESP32 fails to build because it's still on Bluedroid. Was there a particular reason ESP32 was not also moved to NimBLE?

[me-no-dev]

@cpatulea the reason is that nimble does not support classic BT, so if we move ESP32 to nimble, all classic BT functionality is lost

[vidplace7]

@cpatulea the reason is that nimble does not support classic BT, so if we move ESP32 to nimble, all classic BT functionality is lost

@me-no-dev @lucasssvaz This is causing a real headache in our transition from Arduino 2.x to 3.x at the Meshtastic project 😅.
We support (almost) every ESP32 variant, and the nimble-based BLE implementation is working great on everything but ESP32 😬.

Maintaining two codepaths in Meshtastic (nimble + bluedroid) strictly for original-ESP32 support is a pretty large development burden.

Would you consider building two "ESP32" arduino-lib variants, one for nimble and one for bluedroid? Then projects that only use the nimble implementation, like ours, could more easily build.

Example vidplace7@baeadde

Our project seems to be faced now with the decision "port to bluedroid just for original ESP32" or "build our own arduino-lib with nimble enabled, override the bundled lib from pioarduino"

[me-no-dev]

@vidplace7 I am guessing you are not using the Arduino API for BLE? If you were using it, it would have worked regardless of the backend.

[me-no-dev]

another question: how are you building your firmwares? Are you using Arduino CLI or pioarduino?

[vidplace7]

I am guessing you are not using the Arduino API for BLE? If you were using it, it would have worked regardless of the backend.

We are using the integrated Arduino BLE API + some nimble-specific pieces. (Previously we were using h2zero/nimble-arduino)
You can see our 2.x -> 3.x transition work here meshtastic/firmware#9122
see src/nimble/NimbleBluetooth.cpp

another question: how are you building your firmwares? Are you using Arduino CLI or pioarduino?

Currently building with pioarduino.

[cpatulea]

I tried to build using Arduino BLE API and let the ESP32 host be Bluedroid. As vidplace7 said, there are some nimble-specific parts. I added #ifdefs for those. But there is more. For example BLEServer::updateConnParams has different signatures:

BLEServer.h
174:  void updateConnParams(esp_bd_addr_t remote_bda, uint16_t minInterval, uint16_t maxInterval, uint16_t latency, uint16_t timeo>
199:  void updateConnParams(uint16_t conn_handle, uint16_t minInterval, uint16_t maxInterval, uint16_t latency, uint16_t timeout);

In general, connection handles are different (esp_bd_addr_t vs uint16_t) and we store this and pass it around.

So Arduino BLE API does not fully hide the host stack from us. Maybe it possible to keep pushing further with ifdefs. But it's not obviously easy so far.

[lucasssvaz]

@vidplace7 @cpatulea The BLE library already dynamically selects what needs to be compiled based on what is enabled on the static binaries. For example: https://github.com/espressif/arduino-esp32/blob/master/libraries/BLE/src/BLEServer.h#L170-L201

This means you can use the pioarduino's hybrid compilation to override the settings for ESP32, enable NimBLE and use the library without much of a hassle. I haven't tested using ESP32 with NimBLE but there is nothing SoC dependent on the library itself, so it shouldn't be an issue unless there is some weird edge case.

Maybe @Jason2866 can help.

[lucasssvaz]

Also, I am in the middle of a BLE refactor and modernization that will make the API fully stack-agnostic. But it will be available only in v4.0

[me-no-dev]

Hybrid compilation with pioarduino was what I was going to suggest also. It allows you to change the sdkconfig and will recompile the libs automatically. @Jason2866 to the rescue, please :)

[Jason2866]

HybridCompile may work. Needs a test. BT generates maaany include files. This is the problematic part with HybridCompile. The used include files are not changed. If NimBLE and Bluedroid use very different includes this is a mess to fix (needs runtime patching of pioarduino-build.py). Runtime patching of pioarduino-build.py is already done for small fixes / changes. For BT i think it is problematic, if BT include changes in IDF this patch part needs adopting.

But why not building Arduino as a component of IDF? With pioarduino this is easy. Just the initial setup of sdkconfig needs a bit thinkering (the Arduino sdkconfig can be used as boilerplate). The "price" is a bit longer compile time.
Imho the project will have more advantages from this. Settings can be tuned for the project needs.