[BUG]: `load_checks` fails when fully qualified table name includes back quote

[taylorsweetman]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When the fully qualified path to a table needs to include a backtick due to the inclusion of a character such as -, the table is unable to be loaded for the list of checks.

This happens because the workspace client checks for the existence of the table here. The Databricks SDK for tables.exists enforces NOT using backticks. However, if you omit the backticks such that the workspace client can confirm the existence of the table, then when the table is loaded into a spark df here, the table cannot be loaded because spark DOES enforce the usage of backticks.

Expected Behavior

Backticks can be included in the location argument of calls to the load_checks function when using a TableChecksStorageConfig without throwing any errors.

Steps To Reproduce

1. Create a table with the fully qualified name of example-catalog.example-schema.example-table (notice the inclusion of the hyphens)
2. Attempt to load checks from this table:

checks = dq_engine.load_checks(config=TableChecksStorageConfig(location="`example-catalog`.`example-schema`.`example-table`"))

3. Confirm that DQX throws an exception about the table not existing in the workspace (even though it was created in step 1 above).

Cloud

Azure

Operating System

Windows

Relevant log output

When including the backticks:

NotFound: Checks table `example-catalog`.`example-schema`.`example-table` does not exist in the workspace


When excluding the backticks:

ParseException: [INVALID_IDENTIFIER] ...

[cornzyblack]

Hi @taylorsweetman and @mwojtyczka ,

I've been looking into this and can confirm the behavior is as described.

Spark needs those backticks (e.g., `catalog`.`schema`.`table-name`) to correctly parse identifiers with hyphens or special characters. However, the SDK tables.exists treats the input as a literal string for a URL path; it doesn't recognize SQL-style escaping, so it returns False when it sees backticks.

I've been testing this and I proposed the following,

I used explicit URL encoding to prevent the underlying REST call from the SDK self.ws.tables.exists(api_ready_location).table_exists from becoming malformed.

    def load(self, config: TableChecksStorageConfig) -> list[dict]:
        """
        Load checks (dq rules) from a Delta table in the workspace.

        Args:
            config: configuration for loading checks, including the table location and run configuration name.

        Returns:
            list of dq rules or raise an error if checks table is missing or is invalid.

        Raises:
            NotFound: if the table does not exist in the workspace
        """
        logger.info(f"Loading quality rules (checks) from table '{config.location}'")
        # API Path: Remove SQL backticks and URL-encode for the SDK
        api_ready_location = quote(config.location.replace("`", ""))
        
        if not self.ws.tables.exists(api_ready_location).table_exists:
            raise NotFound(f"Checks table {config.location} does not exist in the workspace")
        rules_df = self.spark.read.table(config.location)
        return serialize_checks_from_dataframe(rules_df, run_config_name=config.run_config_name) or []

I have added a test as well, just incase.

@mwojtyczka I have a PR opened for this fix and an integration test to verify the hyphenated/special-character table name scenarios.

[mwojtyczka]

Awesome, thank you for reporting this and the PR. I will review.