Skip to content

RAD-207: Update L1 and L2 Schema#574

Merged
PaulHuwe merged 14 commits into
spacetelescope:mainfrom
PaulHuwe:RAD-207_L1L2Updates
Apr 11, 2025
Merged

RAD-207: Update L1 and L2 Schema#574
PaulHuwe merged 14 commits into
spacetelescope:mainfrom
PaulHuwe:RAD-207_L1L2Updates

Conversation

@PaulHuwe
Copy link
Copy Markdown
Collaborator

@PaulHuwe PaulHuwe commented Apr 9, 2025

Resolves RAD-207

Closes #561 #568

This PR adjusts several RTB directed L1 & L2 metadata schema changes.

Tasks

  • Update or add relevant rad tests.
  • Update relevant docstrings and / or docs/ page.
  • Does this PR change any schema files?
    • Schema changes were discussed at RAD Review Board meeting.
  • Does this PR change any API used downstream? (If not, label with no-changelog-entry-needed.)
News fragment change types:
  • changes/<PR#>.feature.rst: new feature
  • changes/<PR#>.bugfix.rst: fixes an issue
  • changes/<PR#>.doc.rst: documentation change
  • changes/<PR#>.removal.rst: deprecation or removal of public API
  • changes/<PR#>.misc.rst: infrastructure or miscellaneous change

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.69%. Comparing base (7b0130c) to head (1929993).
Report is 126 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #574   +/-   ##
=======================================
  Coverage   96.69%   96.69%           
=======================================
  Files           4        4           
  Lines         272      272           
=======================================
  Hits          263      263           
  Misses          9        9           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@schlafly schlafly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment for one missing description inline. We should develop a system for better reviewing these PRs; maybe you could post a diff between the new and old files with the PRs so that we can see only the differences? Thanks for pushing this out quickly; when the corresponding rdm changes are in I can try to get this going through romanisim.

Comment thread src/rad/resources/schemas/guidestar-1.1.0.yaml
@PaulHuwe PaulHuwe marked this pull request as ready for review April 9, 2025 19:24
@PaulHuwe
Copy link
Copy Markdown
Collaborator Author

PaulHuwe commented Apr 9, 2025

Regression test is running here:
https://github.com/spacetelescope/RegressionTests/actions/runs/14365168010

We expect some failures.

Copy link
Copy Markdown
Collaborator

@braingram braingram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the changes from #571 and #570 were largely lost in the version bumps. Would you double check the changes from these PRs make it into the new schemas?

Comment thread src/rad/resources/schemas/l2_cal_step-1.1.0.yaml
Comment thread src/rad/resources/schemas/exposure-1.1.0.yaml Outdated
Comment thread src/rad/resources/schemas/exposure-1.1.0.yaml Outdated
source:
origin: TBD
archive_catalog:
datatype: nvarchar(max)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps an archive representative could confirm if this the right datatype?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrookens ?
Note, I set this to nvarchar(max), which is used elsewhere, due to the request:
** Note for Archive Catalog / Ingest: this is potentially a very long string. Unsure of setting a character limit for this. RMO rejected the idea of using bit values and preferred a comma-separated string, e.g., "ERR1, ERR2, ERR3, ..., ERRN".

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be changed in the archive catalog as well. From JWST, we inherited a boolean, but after discussion with RMO and SE re: L0 -> L1 processing, we decided to repurpose this to be more informational than just problem = True/False.

Comment thread src/rad/resources/schemas/guidestar-1.1.0.yaml Outdated
title: Guide Window Mode
description: |
Type of guide window mode used during the exposure.
Type of guide window mode used during the exposure. This is based on
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this but did want to note that this is a change to a released schema. I highly doubt anything is depending on this description.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RMO and SE requested we add the information to the schemas to clarify the type of information (planned vs executed). That was the motivation.

Comment thread src/rad/resources/schemas/visit-1.1.0.yaml Outdated
Comment thread src/rad/resources/schemas/visit-1.1.0.yaml Outdated
Comment thread src/rad/resources/schemas/visit-1.1.0.yaml Outdated
Comment thread src/rad/resources/schemas/segmentation_map-1.1.0.yaml Outdated
@schlafly
Copy link
Copy Markdown
Collaborator

On the topic of improving the workflow for updating schemas in the new versioned model, what if we started each set of schema changes with a PR that does nothing but copy the existing schemas over and bump the version numbers, for schemas that will be touched? Then the git diffs would be useful?

@braingram
Copy link
Copy Markdown
Collaborator

On the topic of improving the workflow for updating schemas in the new versioned model, what if we started each set of schema changes with a PR that does nothing but copy the existing schemas over and bump the version numbers, for schemas that will be touched? Then the git diffs would be useful?

I opened an issue to discuss this. #577 Please feel free to add to it. If folks are available we can have a recap (perhaps once the intermediate release is out) to decide on and schedule improvements.

WilliamJamieson added a commit to WilliamJamieson/rad that referenced this pull request Apr 10, 2025
WilliamJamieson added a commit to WilliamJamieson/rad that referenced this pull request Apr 11, 2025
WilliamJamieson added a commit to WilliamJamieson/rad that referenced this pull request Apr 11, 2025
WilliamJamieson added a commit to WilliamJamieson/rad that referenced this pull request Apr 11, 2025
@PaulHuwe PaulHuwe requested review from braingram and schlafly April 11, 2025 16:45
Copy link
Copy Markdown
Collaborator

@braingram braingram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 questions about new origin changes.

Comment thread src/rad/resources/schemas/exposure-1.1.0.yaml
Comment thread src/rad/resources/schemas/wfi_mode-1.0.0.yaml Outdated
Comment thread src/rad/resources/schemas/exposure-1.1.0.yaml
Copy link
Copy Markdown
Collaborator

@braingram braingram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@PaulHuwe PaulHuwe merged commit f6ae395 into spacetelescope:main Apr 11, 2025
13 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update L1 and L2 Schema

4 participants