feat(data): add SO-101 operator view - #1451
Draft
Ali Zaidi (akzaidi) wants to merge 27 commits into
Draft
Ali Zaidi (akzaidi) wants to merge 27 commits into
Ali Zaidi (akzaidi) wants to merge 27 commits into
Conversation
… panels to dataviewer - add Episode Analyzer tab, motion metrics, and analysis records - add per-episode language instructions in LeRobot v3 task format - add foldable right-pane sections persisted across episodes - import analysis fields as filterable labels; add VLM labeling CLI ✨ - Generated by Copilot
🧪 - Generated by Copilot
- correct spelling and formatting findings - cover analysis label import behavior and no-op paths - restore backend coverage above the required threshold ✅ - Generated by Copilot
- cover Azure label storage factory wiring - verify nested dataset label path resolution ✅ - Generated by Copilot
…/episode-analysis
…resolution - explicitly use IPv4 for backend health check - resolve VLM_JUDGE_ENABLED and VLM_JUDGE_BACKEND from .env if not set - improve DATA_DIR resolution logic from .env 🔧 - Generated by Copilot
…y rollout - add operator-worker uv subproject for acquisition, calibration, teleoperation, recording, and watchdog control - add backend operator package with authorization, host lease, preflight, worker client, and streaming router - add frontend operator workspace with session config, camera preview, telemetry and trajectory plots - add end-effector trajectory plots and three.js dependency for the episode viewer - add operator-view design spec and wire worker into viewer build, lint, test, and validate scripts 🤖 - Generated by Copilot
- add resumable VLM labeling and persisted analysis output - preserve label edits and expose safe import controls - remove stale judge polling and expand behavioral coverage 🔧 - Generated by Copilot
🎨 - Generated by Copilot
🔀 - Generated by Copilot
🔄 - Generated by Copilot
🧪 - Generated by Copilot
🔀 - Generated by Copilot
🧪 - Generated by Copilot
Contributor
Dependency ReviewThe following issues were found:
Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. License Issuesdata-management/viewer/operator-worker/uv.lock
OpenSSF ScorecardScorecard details
Scanned Files
|
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (71.96%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1451 +/- ##
==========================================
- Coverage 88.10% 87.67% -0.43%
==========================================
Files 279 302 +23
Lines 23203 25884 +2681
Branches 3199 3752 +553
==========================================
+ Hits 20444 22695 +2251
- Misses 2071 2337 +266
- Partials 688 852 +164
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Ali Zaidi (akzaidi)
marked this pull request as draft
September 2, 2026 20:41
- separate simulated worker startup and command timeouts - close CodeQL findings and add operator behavior coverage - apply Ruff formatting and cspell vocabulary ✅ - Generated by Copilot
- cover runtime, nonce, torque, rate, and telemetry contracts - raise backend coverage to 90.26 percent 🧪 - Generated by Copilot
Collaborator
|
Could we please add a user documentation update explaining how to use the features? |
- consume recorded per-frame Cartesian poses - restrict SO-101 fallback to recognized joint schemas - auto-frame trajectories across workspace scales 📈 - Generated by Copilot
- add HDF5 judge video support and cache coverage - fix simulated preflight, camera polling, and reload shutdown - align VLM dependencies and portable SO-101 profile paths - document setup, safety, trajectories, and judging 🤖 - Generated by Copilot
- apply the canonical Ruff 0.15.21 formatter - preserve operator behavior and test semantics 🎨 - Generated by Copilot
- ignore zero-byte LeRobot placeholders during discovery - preserve diagnostics for nonempty malformed metadata 🧹 - Generated by Copilot
- declare the static FFmpeg runtime dependency - verify bundled binary resolution 🎞️ - Generated by Copilot
- retain current dependency and security pins - preserve bundled FFmpeg episode clipping 🔀 - Generated by Copilot
- align actuator array with Taplo indentation 🎨 - Generated by Copilot
- expose saved calibration evidence without motion - validate motor mappings and signed encoder offsets - cover fail-closed startup and document check limits 🔎 - Generated by Copilot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
PR Soundtrack: Sade - Smooth Operator + Red Eye Outro
Description
Add an SO-101 operator workflow to the dataviewer for live teleoperation, demonstration recording, and bounded policy rollouts.
The change includes:
This branch includes the episode-analysis changes merged in #1234.
Related: #1234
Type of Change
Component(s) Affected
prerequisites- Azure subscription setupterraform- Terraform infrastructuresetup- OSMO control plane / Helmworkflows- Training and evaluation workflowstraining- Training pipelines and scriptsviewer- Dataviewer backend, frontend, and operator workerdocs/- Operator-view design specificationTesting Performed
planreviewed (no unexpected changes)applytested in dev environmentsmoke_test_azure.py)npm run buildnpm run validateDocumentation Impact
Bug Fix Checklist
Checklist