Skip to content

fix(data): improve Viewer accessible interactions - #1641

Open
Bill Berry (WilliamBerryiii) wants to merge 6 commits into
accessibility/viewer-data-integrityfrom
accessibility/viewer-interactions
Open

Bill Berry (WilliamBerryiii) wants to merge 6 commits into
accessibility/viewer-data-integrityfrom
accessibility/viewer-interactions

Conversation

@WilliamBerryiii

Copy link
Copy Markdown
Member

Pull Request

Description

Improve Dataset Viewer accessibility across keyboard selection, pointer cancellation, focus visibility, language validation, status announcements, and generated API documentation. Add a 28-case semantic Playwright manifest and focused component coverage for the changed interaction behavior.

Closes #1636

Type of Change

  • 🐛 Bug fix (non-breaking change fixing an issue)
  • ✨ New feature (non-breaking change adding functionality)
  • 💥 Breaking change (fix or feature causing existing functionality to change)
  • 📚 Documentation update
  • 🏗️ Infrastructure change (Terraform/IaC)
  • ♻️ Refactoring (no functional changes)

Component(s) Affected

  • infrastructure/terraform/prerequisites/ - Azure subscription setup
  • infrastructure/terraform/ - Terraform infrastructure
  • infrastructure/setup/ - OSMO control plane / Helm
  • workflows/ - Training and evaluation workflows
  • training/ - Training pipelines and scripts
  • docs/ - Documentation

Dataset Viewer frontend, Viewer specifications, and generated API documentation behavior are affected.

Testing Performed

  • Terraform plan reviewed (no unexpected changes)
  • Terraform apply tested in dev environment
  • Training scripts tested locally with Isaac Sim
  • OSMO workflow submitted successfully
  • Smoke tests passed (smoke_test_azure.py)

Targeted local validation:

  • npm --prefix data-management/viewer run validate:frontend: typecheck passed; ESLint reported 0 errors and 38 existing warnings; 1,672 tests passed across 146 files.
  • Viewer accessibility Playwright suite: all 28 semantic cases passed on the integrated candidate.
  • git diff --check origin/main...accessibility/viewer-interactions: passed.

Documentation Impact

  • No documentation changes needed
  • Documentation updated in this PR
  • Documentation issue filed

Bug Fix Checklist

Complete this section for bug fix PRs. Skip for other contribution types.

  • Linked to issue being fixed
  • Regression test included, OR
  • Justification for no regression test:

Checklist

- expose state, focus, status, and cancellation behavior
- align language validation and generated API docs
- preserve responsive interaction semantics

♿ - Generated by Copilot
- verify keyboard, status, pointer, language, and adaptive behavior
- cover generated API documentation routes

♿ - Generated by Copilot
- add semantic Playwright inventory and deterministic fixtures
- exercise adaptive Viewer, Swagger, and ReDoc workflows
- pin the browser test runtime

♿ - Generated by Copilot
…/viewer-interactions

# Conflicts:
#	data-management/viewer/frontend/src/components/__tests__/LabelPanel.test.tsx
#	data-management/viewer/frontend/src/components/annotation-panel/LabelPanel.tsx
#	data-management/viewer/specifications/language-tag-corpus.json
@WilliamBerryiii
Bill Berry (WilliamBerryiii) changed the base branch from main to accessibility/viewer-data-integrity September 22, 2026 03:31
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 241ff07.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

OpenSSF Scorecard

PackageVersionScoreDetails
npm/@axe-core/playwright 4.13.0 🟢 6.1
Details
CheckScoreReason
Security-Policy🟢 10security policy file detected
Code-Review🟢 10all changesets reviewed
Maintained🟢 1023 commit(s) and 12 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 10no binaries found in the repo
Packaging⚠️ -1packaging workflow not detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 7dependency not pinned by hash detected -- score normalized to 7
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 1branch protection is not maximal on development and all release branches
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
npm/@playwright/test 1.61.1 🟢 6.5
Details
CheckScoreReason
Maintained🟢 1030 commit(s) out of 30 and 6 issue activity out of 30 found in the last 90 days -- score normalized to 10
Code-Review🟢 9GitHub code reviews found for 29 commits out of the last 30 -- score normalized to 9
CII-Best-Practices⚠️ 0no badge detected
Vulnerabilities🟢 10no vulnerabilities detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 10security policy file detected
License🟢 10license file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1no published package detected
Token-Permissions⚠️ 0non read-only tokens detected in GitHub workflows
Binary-Artifacts🟢 6binaries present in source code
Dependency-Update-Tool🟢 10update tool detected
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Branch-Protection🟢 3branch protection is not maximal on development and all release branches
npm/axe-core 4.13.0 🟢 6.6
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Maintained🟢 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Security-Policy🟢 10security policy file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Binary-Artifacts🟢 10no binaries found in the repo
Pinned-Dependencies🟢 7dependency not pinned by hash detected -- score normalized to 7
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Branch-Protection🟢 4branch protection is not maximal on development and all release branches
Fuzzing⚠️ 0project is not fuzzed
Packaging🟢 10packaging workflow detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
npm/fsevents 2.3.2 🟢 3.1
Details
CheckScoreReason
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Code-Review🟢 3Found 7/22 approved changesets -- score normalized to 3
Binary-Artifacts🟢 10no binaries found in the repo
Packaging⚠️ -1packaging workflow not detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Security-Policy⚠️ 0security policy file not detected
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
npm/playwright 1.61.1 🟢 6.5
Details
CheckScoreReason
Maintained🟢 1030 commit(s) out of 30 and 6 issue activity out of 30 found in the last 90 days -- score normalized to 10
Code-Review🟢 9GitHub code reviews found for 29 commits out of the last 30 -- score normalized to 9
CII-Best-Practices⚠️ 0no badge detected
Vulnerabilities🟢 10no vulnerabilities detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 10security policy file detected
License🟢 10license file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1no published package detected
Token-Permissions⚠️ 0non read-only tokens detected in GitHub workflows
Binary-Artifacts🟢 6binaries present in source code
Dependency-Update-Tool🟢 10update tool detected
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Branch-Protection🟢 3branch protection is not maximal on development and all release branches
npm/playwright-core 1.61.1 🟢 6.5
Details
CheckScoreReason
Maintained🟢 1030 commit(s) out of 30 and 6 issue activity out of 30 found in the last 90 days -- score normalized to 10
Code-Review🟢 9GitHub code reviews found for 29 commits out of the last 30 -- score normalized to 9
CII-Best-Practices⚠️ 0no badge detected
Vulnerabilities🟢 10no vulnerabilities detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 10security policy file detected
License🟢 10license file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1no published package detected
Token-Permissions⚠️ 0non read-only tokens detected in GitHub workflows
Binary-Artifacts🟢 6binaries present in source code
Dependency-Update-Tool🟢 10update tool detected
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Branch-Protection🟢 3branch protection is not maximal on development and all release branches

Scanned Files

  • data-management/viewer/frontend/package-lock.json

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.83%. Comparing base (566e376) to head (241ff07).

Additional details and impacted files
@@                           Coverage Diff                           @@
##           accessibility/viewer-data-integrity    #1641      +/-   ##
=======================================================================
- Coverage                                87.84%   87.83%   -0.01%     
=======================================================================
  Files                                      281      281              
  Lines                                    23534    23508      -26     
  Branches                                  3193     3188       -5     
=======================================================================
- Hits                                     20673    20648      -25     
+ Misses                                    2150     2149       -1     
  Partials                                   711      711              
Flag Coverage Δ *Carryforward flag
go 100.00% <ø> (ø)
pester 86.75% <ø> (ø) Carriedforward from 566e376
pytest-data-pipeline 100.00% <ø> (ø) Carriedforward from 566e376
pytest-dataviewer 88.79% <ø> (ø)
pytest-dm-tools 100.00% <ø> (ø) Carriedforward from 566e376
pytest-evaluation 95.40% <ø> (ø)
pytest-fuzz 3.76% <ø> (ø)
pytest-inference 100.00% <ø> (ø) Carriedforward from 566e376
pytest-shared-ci 100.00% <ø> (ø) Carriedforward from 566e376
pytest-training 92.60% <ø> (ø) Carriedforward from 566e376
vitest 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376
vitest-app 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376
vitest-components 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376
vitest-features 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376
vitest-lib 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376
vitest-state 85.27% <ø> (-0.05%) ⬇️ Carriedforward from 566e376

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
data-management/viewer/frontend/src/App.tsx 77.77% <ø> (ø)
...end/src/components/annotation-panel/LabelPanel.tsx 79.34% <ø> (ø)
...nts/annotation-panel/LanguageInstructionWidget.tsx 88.23% <ø> (-0.18%) ⬇️
...ponents/annotation-panel/ObjectDetectionWidget.tsx 65.55% <ø> (ø)
...nnotation-workspace/AnnotationWorkspaceContent.tsx 80.00% <ø> (ø)
...tion-workspace/AnnotationWorkspacePlaybackCard.tsx 60.19% <ø> (ø)
...annotation-workspace/AnnotationWorkspaceTopBar.tsx 100.00% <ø> (ø)
...ion-workspace/AnnotationWorkspaceTrajectoryTab.tsx 75.00% <ø> (ø)
...-workspace/useAnnotationWorkspaceEpisodeActions.ts 72.05% <ø> (ø)
...tation-workspace/useAnnotationWorkspacePlayback.ts 86.07% <ø> (-0.29%) ⬇️
... and 19 more
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Dataset Viewer accessible interactions

2 participants