Approach for validation of SSC metadata

[stscijgbot-rstdms]

Issue RAD-225 was created on JIRA by Christopher Arnold:

The SSC pipeline devs have reached out asking how they can validate their JSON metadata objects against the RAD schema, for a given SSC file type.  We are looking for the best approach to accomplish this- an example module or script would be helpful.

Inputs:

• A top level SSC RAD yaml file, such as https://github.com/spacetelescope/rad/blob/main/latest/SSC/MSOS/wfi_microlensing_light_curve_catalog_level_4.yaml
• A JSON object representing example SSC metadata

Outputs:

• Whether or not the JSON object passed the JSON schema validation for that file type (and some logging, if issues are found)

[stscijgbot-rstdms]

Comment by Nadia Dencheva on JIRA:

Christopher Arnold Just to clarify, are you looking to validate the manifest or the actual data file? Is there an example file (that the JSON object represents) that you can share?

[stscijgbot-rstdms]

Comment by Christopher Arnold on JIRA:

Hi, SSC is looking to validate the manifest (metadata JSON object) only.  Here is an example JSON object: [^cgi_level_1_metadata_example.json]which we would want validated against https://github.com/spacetelescope/rad/blob/main/latest/SSC/CGI/cgi_level_1.yaml](https://github.com/spacetelescope/rad/blob/main/latest/SSC/CGI/cgi_level_1.yaml)

 

[stscijgbot-rstdms]

Comment by Brett Graham on JIRA:

Thanks for sharing the file. To clarify, is the expectation that the schemas validate the contents of this file? If so they don't currently agree.

The manifest contains a dictionary that is 1 level deep (only has top level keys). Of note is that "meta" is not one of these keys.

On the other hand the schema only checks:

• the provided instance in an object/dictionary (this passes)
• if the object contains a "meta" key additional checks would be made, however since there is no "meta" key, no additional checks are performed

To summarize, the example manifest passes validation because the schema only checks that the provided instance is an object/dictionary. The current schema also says that an empty dictionary is valid.

[stscijgbot-rstdms]

Comment by Christopher Arnold on JIRA:

Brett Graham The 'meta' nesting in the RAD schemas for SSC files is wrong (I misunderstood its use when I first created the RAD schemas). On the next round of SSC RAD updates, I will do a PR to remove the meta nesting. So assuming the meta is removed, yes the expectation is the schemas validate the contents of this file.

[braingram]

Thanks. Here's some code that will allow you to test with the current schemas (by nesting the manifest under a "meta" key):

import json

import asdf
import asdf_standard

# add ssc resources
resources = asdf_standard.DirectoryResourceMapping(
    "src/rad/resources/schemas/SSC",
    "asdf://stsci.edu/datamodels/roman/schemas/SSC/",
    recursive=True,
)

asdf.get_config().add_resource_mapping(resources)

asdf.AsdfFile(
    {"meta": json.load(open("/Users/bgraham/Downloads/cgi_level_1_metadata_example.json"))},
    custom_schema="asdf://stsci.edu/datamodels/roman/schemas/SSC/CGI/cgi_level_1-1.0.0",
).validate()

The code assumes it's being run in a local clone of rad (see the "src/rad/resources/..." argument to DirectoryResourceMapping).

When I run this I get a validation error "Failed validating 'tag' in schema['properties']['meta']['allOf'][0]['properties']['file_date']:" due to file_date being a non-tagged string.

[ketozhang]

Would it be much to put the resource mapping into the RAD extension? It would be nice to simplify it to:

import asdf

metadata: dict = {...}
asdf.AsdfFile(
    {"meta": metadata}, 
    custom_schema="asdf://stsci.edu/datamodels/roman/schemas/SSC/CGI/cgi_level_1-1.0.0"
).validate()

Alternatively

from asdf.schema import load_schema, validate

metadata: dict = {...}
validate(
    {"meta": metadata},   # type check? probably not valid
    schema=load_schema("asdf://stsci.edu/datamodels/roman/schemas/SSC/CGI/cgi_level_1-1.0.0")
)

[braingram]

Thanks. Since these schemas are not versioned and not used for any ASDF file they were removed from the resource registration in #686

I do not think we want to encourage the use of these schemas outside of this very limited context (where we can register them as a resource mapping).

[cjarnold]

This commit on the CGI schemas removes the need to nest the JSON object in "meta", like:

{"meta": json.load(open("/Users/bgraham/Downloads/cgi_level_1_metadata_example.json"))}

Instead, do

json.load(open("/Users/bgraham/Downloads/cgi_level_1_metadata_example.json"))

The commit also fixes the datetime validation.
I will do the same for the MSOS and GDPS schemas.

[ketozhang]

@braingram I understand the intention—this repository does hosts both public facing and internal facing data models.

Will the schemas be included in the Python package resources (i.e., can be found with importlib). I am looking at this unit test—but I did not double check if it's made available through standard install (no extras, no dependency groups):

rad/tests/test_schemas.py

Lines 502 to 506 in d19a753

	with asdf.config_context() as config:
	resource_mapping = asdf.resource.DirectoryResourceMapping(
	files(resources) / "schemas" / "SSC", "asdf://stsci.edu/datamodels/roman/schemas/SSC/", recursive=True
	)
	config.add_resource_mapping(resource_mapping)

[WilliamJamieson]

All of the schemas including the SSC ones are included in the python package. They are not registered by default with ASDF for the reasons that @braingram detailed.

[braingram]

@ketozhang does this address your concerns?

@cjarnold is the provided example sufficient to close this ticket?

[ketozhang]

Yep that's great! If we can close off with an example that does not require our app to clone this repo then that would be perfect.

[braingram]

Can do. Let me know if you have any questions:

from importlib.resources import files
import json

import asdf
import asdf_standard

# add ssc resources
resources = asdf.resource.DirectoryResourceMapping(
    files("rad") / "resources" / "schemas" / "SSC",
    "asdf://stsci.edu/datamodels/roman/schemas/SSC/",
    recursive=True
)

asdf.get_config().add_resource_mapping(resources)

asdf.AsdfFile(
    {"meta": json.load(open("/Users/bgraham/Downloads/cgi_level_1_metadata_example.json"))},
    custom_schema="asdf://stsci.edu/datamodels/roman/schemas/SSC/CGI/cgi_level_1-1.0.0",
).validate()