Skip check evaluation if columns or filter cannot be resolved in the input DataFrame

docs/dqx/docs/guide/quality_checks_apply.mdx

@@ -14,6 +14,7 @@ You can apply quality checks to your data using the following approaches:
 * [No-code approach (Workflows)](#no-code-approach-workflows): you can apply checks for data at-rest using workflows if DQX is installed in the workspace as a tool. This does not require code-level integration and is suitable for data already stored in Delta tables or files.
 
 DQX offers a set of predefined built-in quality rules (checks) as described [here](/docs/guide/quality_checks_definition).
+All DQX rules are evaluated independently and in parallel. If one rule fails, the others are still processed.
 
 ## Programmatic approach
 
@@ -30,7 +31,8 @@ Checks can be applied to the input data by one of the following methods of the `
 
 You can see the full list of `DQEngine` methods [here](/docs/reference/engine/#dqx-engine-methods).
 
-The engine will raise an error if you try to apply checks with invalid definition.
+The engine ensures that the specified `column`, `columns`, or `filter` fields can be resolved in the input DataFrame. If any of these fields are invalid, the check evaluation is skipped, and the results include the check failure with a message identifying the invalid fields.
+The engine will raise an error if you try to apply checks with invalid definition (e.g. wrong syntax).
 In addition, you can also perform a standalone syntax validation of the checks as described [here](/docs/guide/quality_checks_definition#validating-syntax-of-quality-checks).
 
 You can apply quality checks to streaming pipelines using the same methods as for batch processing.

docs/dqx/docs/guide/quality_checks_definition.mdx

@@ -553,17 +553,17 @@ Checks defined in delta tables are supported by the DQX workflows.
 
 The Delta table used to store checks has the following structure:
 
-| Column               | Type                  | Description                                                                                                                              |
-| -------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
-| `name`               | `string`              | Name to use for the check. Optional.                                                                                                     |
-| `criticality`        | `string`              | Either `"error"` (rows go only to "bad" dataset) or `"warn"` (rows go to both "good" and "bad"). Defaults to `"error"`.                  |
-| `check`              | `struct`              | Defines the DQX check to apply.                                                                                                          |
-| └─ `function`        | `string`              | Name of the DQX check function to apply.                                                                                                 |
-| └─ `arguments`       | `map<string, string>` | Key-value pairs passed as keyword arguments to the function, stored as JSON string so quotes for string arguments must be escaped.       |
-| └─ `for_each_column` | `array<string>`       | (Optional) List of column names or expressions to which the check will be applied individually.                                          |
+| Column               | Type                  | Description                                                                                                                                               |
+| -------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `name`               | `string`              | Name to use for the check. Optional.                                                                                                                      |
+| `criticality`        | `string`              | Either `"error"` (rows go only to "bad" dataset) or `"warn"` (rows go to both "good" and "bad"). Defaults to `"error"`.                                   |
+| `check`              | `struct`              | Defines the DQX check to apply.                                                                                                                           |
+| └─ `function`        | `string`              | Name of the DQX check function to apply.                                                                                                                  |
+| └─ `arguments`       | `map<string, string>` | Key-value pairs passed as keyword arguments to the function, stored as JSON string so quotes for string arguments must be escaped.                        |
+| └─ `for_each_column` | `array<string>`       | (Optional) List of column names or expressions to which the check will be applied individually.                                                           |
 | `filter`             | `string`              | (Optional) Spark SQL expression to filter rows to which the check is applied, e.g. `"business_unit = 'Finance'"`. The check function will run only on the rows matching the filter condition. The condition can reference any column of the validated dataset, not only the one where you apply the check function. |
-| `run_config_name`    | `string`              | Name of the run config name. Could be any string such as workflow name. Useful for selecting applicable checks. Defaults to `"default"`. |
-| `user_metadata`      | `map<string, string>` | (Optional) Custom metadata to add to any row-level warnings or errors generated by the check.                                            |
+| `run_config_name`    | `string`              | Name of the run config name. Could be any string such as input table or job name (use "default" if not provided). Useful for selecting applicable checks. |
+| `user_metadata`      | `map<string, string>` | (Optional) Custom metadata to add to any row-level warnings or errors generated by the check.                                                             |
 
 Example checks saved in the Delta table:
 ```python
@@ -576,6 +576,8 @@ Example checks saved in the Delta table:
 +------------------------+-------------+------------------------------------------------------------------------------------------------------------------------+----------+-----------------+-------------------------------------------+
 ```
 
+If `run_config_name` is not provided, "default" is used. Typically, the input table or job name is used for run config name to establish a one-to-one mapping between tables or jobs and checks.
+
 ## Validating syntax of quality checks
 
 You can validate the syntax of checks loaded from a storage system or checks defined programmatically before applying them.

docs/dqx/docs/guide/quality_checks_storage.mdx

@@ -18,14 +18,14 @@ Saving and loading methods accept a storage backend configuration as input. The
     * `location`: absolute workspace file path (JSON or YAML).
 * `TableChecksStorageConfig`: Unity Catalog tables. Containing fields:
     * `location`: table fully qualified name.
-    * `run_config_name`: (optional) run configuration name to load (use "default" if not provided).
+    * `run_config_name`: (optional) run configuration name to load (it can be any string), e.g. input table or job name (use "default" if not provided).
     * `mode`: (optional) write mode for saving checks (`overwrite` or `append`, default is `overwrite`). The `overwrite` mode will only replace checks for the specific run config and not all checks in the table.
 * `VolumeFileChecksStorageConfig`: Unity Catalog Volume (JSON/YAML file). Containing fields:
     * `location`: Unity Catalog Volume file path (JSON or YAML).
 * `InstallationChecksStorageConfig`: installation-managed location from the run config, ignores location and infers it from `checks_location` in the run config. Containing fields:
     * `location` (optional): automatically set based on the `checks_location` field from the run configuration.
     * `install_folder`: (optional) installation folder where DQX is installed, only required when custom installation folder is used.
-    * `run_config_name` (optional) - run configuration name to load (use "default" if not provided).
+    * `run_config_name` (optional) - run configuration name to load (it can be any string), e.g. input table or job name (use "default" if not provided).
     * `product_name`: (optional) name of the product (use "dqx" if not provided).
     * `assume_user`: (optional) if True, assume user installation, otherwise global installation (skipped if `install_folder` is provided).
 
@@ -72,18 +72,19 @@ If you create checks as a list of `DQRule` objects, you can convert them to meta
     # save checks as a YAML file in arbitrary workspace location (overwrite the file) using absolute path
     dq_engine.save_checks(checks, config=WorkspaceFileChecksStorageConfig(location="/Shared/App1/checks.yml"))
 
-    # save checks in a Delta table to "default" run config (append checks)
+    # save checks in a Delta table to "default" run config, append checks
     dq_engine.save_checks(checks, config=TableChecksStorageConfig(location="catalog.schema.checks_table", mode="append"))
 
-    # save checks in a Delta table with specific run config for filtering (overwrite checks)
-    dq_engine.save_checks(checks, config=TableChecksStorageConfig(location="catalog.schema.checks_table", run_config_name="workflow_001", mode="overwrite"))
+    # save checks in a Delta table with specific run config for filtering (it can be any string, e.g. input table or job name), overwrite checks
+    dq_engine.save_checks(checks, config=TableChecksStorageConfig(location="catalog.schema.checks_table", run_config_name="main.default.input_table", mode="overwrite"))
 
     # save checks as a YAML in a Unity Catalog Volume location (overwrite the file)
     dq_engine.save_checks(checks, config=VolumeFileChecksStorageConfig(location="/Volumes/dq/config/checks_volume/App1/checks.yml"))
 
     # save checks as a YAML file or table defined in 'checks_location' of the run config
     # only works if DQX is installed in the workspace
-    dq_engine.save_checks(checks, config=InstallationChecksStorageConfig(assume_user=True, run_config_name="default"))
+    # the run config name can be any string, e.g. input table or job name
+    dq_engine.save_checks(checks, config=InstallationChecksStorageConfig(assume_user=True, run_config_name="main.default.input_table"))
     ```
   </TabItem>
   <TabItem value="Workflows" label="Workflows">
@@ -210,15 +211,16 @@ If you create checks as a list of DQRule objects, you can convert them using the
     # load checks from a Delta table and default run config name
     checks: list[dict] = dq_engine.load_checks(config=TableChecksStorageConfig(location="catalog.schema.checks_table"))
 
-    # load checks from a Delta table with specific run config for filtering
-    checks: list[dict] = dq_engine.load_checks(config=TableChecksStorageConfig(location="catalog.schema.checks_table", run_config_name="workflow_001"))
+    # load checks from a Delta table with specific run config for filtering (it can be any string, e.g. input table or job name)
+    checks: list[dict] = dq_engine.load_checks(config=TableChecksStorageConfig(location="catalog.schema.checks_table", run_config_name="main.default.input_table"))
 
     # load checks from a Unity Catalog Volume
     checks: list[dict] = dq_engine.load_checks(config=VolumeFileChecksStorageConfig(location="/Volumes/dq/config/checks_volume/App1/checks.yml"))
 
     # load checks from a file or table defined in the run config ('checks_location' field)
     # only works if DQX is installed in the workspace
-    checks: list[dict] = dq_engine.load_checks(config=InstallationChecksStorageConfig(run_config_name="default"))
+    # the run config name is a string (e.g. input table or job name)
+    checks: list[dict] = dq_engine.load_checks(config=InstallationChecksStorageConfig(run_config_name="main.default.input_table"))
 
     # validate loaded checks
     assert not dq_engine.validate_checks(checks).has_errors

docs/dqx/docs/installation.mdx

@@ -180,7 +180,7 @@ extra_params:                       # <- optional extra parameters to pass to th
 
 run_configs:                        # <- list of run configurations, each run config defines one specific input and output location
 
-- name: default                     # <- unique name of the run config (default used during installation)
+- name: default                     # <- unique name of the run config (it can be any string), e.g. input table or job name (use "default" during installation)
 
   input_config:                     # <- optional input data configuration
     location: s3://iot-ingest/raw   # <- input location of the data (table or cloud path)