Is there an existing issue for this?
Problem statement
Validating ingested JSON is a common requirement for many ETL scenarios. Deserialization and other downstream processing are often broken when JSON is malformed or missing expected fields
Proposed Solution
Add several built-in check functions to validate JSON strings:
is_valid_json to detect the validity of a JSON string
has_json_keys to detect if 1 or more keys are present in a JSON string
has_json_schema to detect if a JSON string matches an expected Spark DDL schema
Additional Context
No response
Is there an existing issue for this?
Problem statement
Validating ingested JSON is a common requirement for many ETL scenarios. Deserialization and other downstream processing are often broken when JSON is malformed or missing expected fields
Proposed Solution
Add several built-in check functions to validate JSON strings:
is_valid_jsonto detect the validity of a JSON stringhas_json_keysto detect if 1 or more keys are present in a JSON stringhas_json_schemato detect if a JSON string matches an expected Spark DDL schemaAdditional Context
No response