Repository navigation
feat(lightdash): emit raw measures and synthesise a row count - #18
Merged
Merged
Conversation
Two gaps that between them left 34 of 38 explores with no metric at all.
RAW MEASURES. The emitter read only t.Metrics, so any ir.Measure no metric
wrapped was dropped silently — along with its backing column, since a column
only appears if something references it. On the benchmark source that meant 22
measures became 9 metrics: clicks and impressions vanished from
obt_marketing_daily entirely, making "how many ad clicks" unanswerable through
a semantic layer whose dbt model declares the column and whose warehouse table
has it.
databricks_metric_view.go already had this pass. Not having it here meant one
IR produced a materially thinner layer for one target than another, which is
the opposite of what an IR is for. Dedup follows that precedent exactly:
raw-vs-metric by name OR rendered aggregate, raw-vs-raw by name only.
A measure whose expr is not a plain column degrades to a note. Without that
guard "case when is_refunded then 1 else 0 end" was emitted as a COLUMN NAME,
and Lightdash would look for a column called that.
ROW COUNT. A table left with no metric now gets <table>_count over its primary
key. Lightdash does not require this — ad-hoc custom metrics work fine against
a dimension-only explore, verified through Lightdash's own MCP path, where a
custom count on dim_product returns 103.
It is a workaround for an upstream bug, and worth being explicit that it is:
given nothing selectable, Lightdash's AI agent invents a metric id, and
MetricQueryBuilder joins the select list with no empty guard, so the
unresolvable field is dropped and the rest compiles to
SELECT
FROM `lightdash`.`fct_order_lines` AS "fct_order_lines"
which ClickHouse rejects (Code 62). Postgres accepts `SELECT FROM t`, which is
why it went unnoticed upstream. That accounted for 9 of 18 failures in a
50-question benchmark run. A real countable metric removes the incentive.
Result on the benchmark source: 38/38 explores now carry metrics (was 4/38),
56 metrics total (was 9).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XdPy5mkvNRuGZ59rShLUag
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gaps that between them left 34 of 38 explores with no metric at all.
Raw measures
The emitter read only
t.Metrics, so anyir.Measureno metric wrapped was dropped silently, along with its backing column (a column only appears if something references it).On the benchmark source that meant 22 measures became 9 metrics.
clicksandimpressionsvanished fromobt_marketing_dailyentirely, making "how many ad clicks" unanswerable through a semantic layer whose dbt model declares the column and whose warehouse table has it.databricks_metric_view.goalready had this pass. Not having it here meant one IR produced a materially thinner layer for one target than another, which is the opposite of what an IR is for. Dedup follows that precedent exactly: raw-vs-metric by name or rendered aggregate, raw-vs-raw by name only.A measure whose expr is not a plain column degrades to a note. Without that guard,
case when is_refunded then 1 else 0 endwas emitted as a column name.Row count
A table left with no metric gets
<table>_countover its primary key.Lightdash does not require this. Ad-hoc custom metrics work fine against a dimension-only explore, verified through Lightdash's own MCP path where a custom count on
dim_productreturns 103. So this is a workaround for an upstream bug, and the code says so.Given nothing selectable, Lightdash's AI agent invents a metric id.
MetricQueryBuilderjoins the select list with no empty guard, so the unresolvable field is dropped and the rest compiles to:ClickHouse rejects that (Code 62). Postgres accepts
SELECT FROM t, which is why it went unnoticed upstream. It accounted for 9 of 18 failures in a 50-question benchmark run. A real countable metric removes the incentive to invent one.Result
Three tests added, golden regenerated. Suite green, vet and gofmt clean.