feat: support metrics-only writes without output or quarantine tables

docs/dqx/docs/guide/quality_checks_apply.mdx

@@ -37,8 +37,8 @@ Checks can be applied to the input data by one of the following methods of the `
 
 You can see the full list of `DQEngine` methods [here](/docs/reference/engine/#dqx-engine-methods).
 
-<Admonition type="info" title="Quarantine-only mode">
-The end-to-end `apply_checks_and_save_in_table` and `apply_checks_by_metadata_and_save_in_table` methods accept either `output_config`, `quarantine_config`, or both. If `output_config` is omitted and `quarantine_config` is provided, only the quarantined table is produced. This applies to both `apply_checks_and_save_in_tables` and `apply_checks_and_save_in_tables_for_patterns`.
+<Admonition type="info" title="Save destinations">
+The end-to-end `apply_checks_and_save_in_table` and `apply_checks_by_metadata_and_save_in_table` methods accept `output_config`, `quarantine_config`, `metrics_config`, or combinations of these. If `output_config` is omitted and `quarantine_config` is provided, only the quarantined table is produced. If only `metrics_config` is provided for batch input, only summary metrics are written and no output or quarantine table is produced. Streaming metrics require an output or quarantine stream to emit the observed metrics. This applies to both `apply_checks_and_save_in_tables` and `apply_checks_and_save_in_tables_for_patterns`.
 </Admonition>
 
 The engine ensures that the specified `column`, `columns`, `filter`, or sql 'expression' fields can be resolved in the input DataFrame. If any of these fields are invalid, the check evaluation is skipped, and the results include the check failure with a message identifying the invalid fields and `skipped=True` in the result struct. You can suppress these entries entirely or identify them downstream — see [Suppressing skipped check entries](/docs/guide/additional_configuration#suppressing-skipped-check-entries).
@@ -1008,6 +1008,13 @@ To enable summary metrics programmatically, create and pass a `DQMetricsObserver
         output_config=OutputConfig(location="catalog.schema.output"),
         metrics_config=OutputConfig(location="catalog.schema.metrics"),
     )
+
+    # Option 5 End-to-End approach: apply quality checks to a batch table and save only summary metrics
+    dq_engine.apply_checks_and_save_in_table(
+        checks=checks,  # or provide checks_location and run_config_name to auto-load from checks storage
+        input_config=InputConfig(location="catalog.schema.input"),
+        metrics_config=OutputConfig(location="catalog.schema.metrics"),
+    )
     ```
   </TabItem>
 </Tabs>

docs/dqx/docs/guide/summary_metrics.mdx

@@ -125,6 +125,7 @@ Metrics are not directly accessible from the returned Spark Observation when dat
 
 End-to-end engine methods for applying checks (e.g. `apply_checks_and_save_in_table`, `apply_checks_by_metadata_and_save_in_table`, `save_results_in_table`, `apply_checks_and_save_in_tables`, `apply_checks_and_save_in_tables_for_patterns`) can write summary metrics into a table automatically using configuration.
 Metrics can be written to a table in batch or streaming. You can write metrics for different datasets or workloads into a common metrics table to track data quality over time centrally.
+For batch inputs, these methods can write summary metrics without writing output or quarantine tables by providing `metrics_config` only.
 
 The `name` specified in the `DQMetricsObserver` is recorded as the `run_name` column in the metrics table. It is recommended to assign a unique `name` to the observer for each job or table to enable efficient filtering when centralizing metrics in a single table.
 
@@ -134,7 +135,7 @@ Writing metrics directly to a results table is not supported for classic compute
 
 #### Writing Metrics to a Table in Batch
 
-Summary metrics can be written to a table when calling `DQEngine` methods to apply checks and write output data. When the input data is read as a batch source, metrics will be collected and written in batch.
+Summary metrics can be written to a table when calling `DQEngine` methods to apply checks and write output data, quarantine data, or metrics only. When the input data is read as a batch source, metrics will be collected and written in batch.
 
 <Tabs>
   <TabItem value="Python" label="Python" default>
@@ -202,6 +203,13 @@ Summary metrics can be written to a table when calling `DQEngine` methods to app
         quarantine_config=quarantine_config,
         metrics_config=metrics_config
     )
+
+    # Option 3: Use End to End method to write summary metrics only for a batch input
+    engine.apply_checks_and_save_in_table(
+        checks=checks,  # or provide checks_location and run_config_name to auto-load from checks storage
+        input_config=input_config,
+        metrics_config=metrics_config
+    )
     ```
   </TabItem>
 </Tabs>
@@ -213,6 +221,7 @@ Summary metrics can also be written in streaming. When the input data is read as
 <Admonition type="warning" title="Supported methods">
 Metrics are not directly accessible from the returned Spark Observation when data is processed with streaming.
 You must use streaming metrics listener or end-to-end methods that persist the output in tables after quality checks are applied (e.g. e.g. `apply_checks_and_save_in_table`, `apply_checks_by_metadata_and_save_in_table`, `save_results_in_table`, `apply_checks_and_save_in_tables`, `apply_checks_and_save_in_tables_for_patterns`).
+Metrics-only streaming writes are not supported because DQX needs an output or quarantine streaming query to emit observed metrics.
 </Admonition>
 
 <Tabs>
@@ -299,7 +308,7 @@ You must use streaming metrics listener or end-to-end methods that persist the o
 
 #### Saving Results and Metrics to a Table
 
-Summary metrics can also be written to a table when calling `save_results_in_table`. After applying checks, pass the Spark Observation and output DataFrame(s) with the appropriate output configuration.
+Summary metrics can also be written to a table when calling `save_results_in_table`. After applying checks, pass the Spark Observation and output DataFrame(s) with the appropriate output configuration. For batch results, you can pass only the Spark Observation and `metrics_config` to write summary metrics without writing row-level output.
 This is supported for both batch and streaming.
 
 <Tabs>
@@ -359,6 +368,14 @@ This is supported for both batch and streaming.
         quarantine_config=quarantine_config,
         metrics_config=metrics_config
     )
+
+    # Write only summary metrics for batch results
+    checked_df, observation = engine.apply_checks(df, checks)
+    checked_df.count()  # Trigger the observation before saving metrics
+    engine.save_results_in_table(
+        observation=observation,
+        metrics_config=metrics_config
+    )
     ```
   </TabItem>
 </Tabs>