feat(cameras): add Orbbec RGBD camera support via pyorbbecsdk2

[orbbecwuxin]

feat(cameras): add Orbbec RGBD camera support via pyorbbecsdk2

Type / Scope

• Type: Feature
• Scope: cameras/orbbec, docs, requirements

Summary / Motivation

Adds full support for Orbbec Gemini series cameras (tested on Gemini 336 / 336L) to LeRobot, mirroring the existing RealSense camera interface. The driver is built on pyorbbecsdk2 (SDK 2.0.18) and uses a background thread for async frame capture. Key capabilities include color+depth stream synchronisation, hardware/software depth-to-color (D2C) alignment with automatic fallback, and multi-camera auto-discovery.

Related issues

• Fixes / Closes: # (if any)
• Related: # (if any)

What changed

• Added src/lerobot/cameras/orbbec/camera_orbbec.py: OrbbecCamera implementation with sync/async read, color+depth frame sync, hardware D2C alignment (auto-falls back to software AlignFilter when unsupported), and multi-format color conversion (RGB/BGR/YUYV/MJPG/NV12/NV21/I420).
• Added src/lerobot/cameras/orbbec/configuration_orbbec.py: OrbbecCameraConfig with D2CMode enum (HARDWARE / SOFTWARE); defaults to SOFTWARE for maximum device compatibility.
• Added src/lerobot/cameras/orbbec/__init__.py: module exports.
• Added tests/cameras/test_orbbec.py: 32 unit tests (mock-based), covering 12 sync/async scenarios.
• Modified src/lerobot/cameras/utils.py, scripts/lerobot_find_cameras.py: register orbbec camera type and lerobot-find-cameras orbbec CLI entry point.
• Modified src/lerobot/scripts/lerobot_calibrate.py, lerobot_record.py, lerobot_teleoperate.py: add OrbbecCameraConfig import.
• Modified pyproject.toml, requirements-ubuntu.txt: add pyorbbecsdk2 as an optional dependency.
• Modified docs/source/cameras.mdx, docs/source/installation.mdx: add Orbbec installation instructions and udev rules.
• Breaking change: none — existing camera interfaces are unaffected.

How was this tested (or how to run locally)

• Tests added: tests/cameras/test_orbbec.py (32 tests, all mock-based)

• Run unit tests:

  pytest -q tests/cameras/test_orbbec.py

• Install Orbbec SDK:

  pip install pyorbbecsdk2
  # Configure udev rules (see docs/source/installation.mdx)

• Hardware validation (2 cameras: Gemini 336 + Gemini 336L, USB 3.2):

  lerobot-find-cameras orbbec
  
  lerobot-teleoperate \
    --robot.type=so101_follower \
    --robot.port=/dev/so101_follower \
    --robot.id=so101_follower \
    --robot.cameras='{"handeye":{"type":"orbbec","index_or_serial_number":"<SN1>","width":640,"height":480,"fps":30,"use_depth":true,"align_depth":true,"d2c_mode":"hardware"},"up":{"type":"orbbec","index_or_serial_number":"<SN2>","width":640,"height":480,"fps":30,"use_depth":true,"align_depth":true,"d2c_mode":"hardware"}}' \
    --teleop.type=so101_leader \
    --teleop.port=/dev/so101_leader \
    --teleop.id=so101_leader \
    --display_data=true

Checklist (required before merge)

• Linting/formatting run (pre-commit run -a)
• All tests pass locally (pytest)
• Documentation updated
• CI is green

Reviewer notes

• Please focus on the hardware D2C fallback logic in _configure_pipeline (src/lerobot/cameras/orbbec/camera_orbbec.py): when the device does not support hardware D2C, the code automatically switches to a software AlignFilter to prevent runtime crashes.
• D2CMode default was changed from HARDWARE to SOFTWARE for broader device compatibility; hardware mode offers lower latency and can be explicitly enabled via config.
• Mock tests cover all public methods; hardware integration tests require manual validation on a real device.
• Anyone in the community is free to review the PR.