Repository navigation
feat(lightdash): Lightdash emitter, merged with main and collision-safe - #17
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refine the spec: derived metrics require refs to emitted same-table simple
metrics (no dangling ${...}); drop time_intervals (no IR interval list);
count(*) degrades.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The design section still described the older flag/identity config; the implementation (and plan) use a semglot.yaml profile field. Match the doc to what was built. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lightdash gives dimensions and metrics one namespace, and every entry in
columns[] becomes a dimension -- including a column the emitter creates
only to host a column-level metric. When a metric carried the same name,
Lightdash kept the dimension and skipped the metric:
obt_marketing_daily: Skipped metric "roas" because a dimension with the
same name exists. Dimensions take priority.
obt_marketing_daily: Skipped metric "attributed_revenue" because a
dimension with the same name exists. Dimensions take priority.
The skip is a line in the deploy log and nothing in the emitted file, so
downstream the metric just looks like it was never defined -- an analytics
agent asking for roas finds only a raw per-row column and answers a
different question. Over the ecommerce marts both collisions were on
obt_marketing_daily: sum(attributed_revenue) named attributed_revenue
(the metric's own backing column is the collision) and the derived
roas = attributed_revenue / ad_spend against a precomputed roas column.
Resolve it the way snowflake-semantic-view and databricks-metric-view
already resolve the same clash: the computed metric is canonical, the
colliding field goes. The colliding column is no longer emitted, and any
metric it hosted is re-homed at model level with an explicit
sql: ${TABLE}.<col> -- the same aggregation, spelled out instead of
inferred from the column it hung on. Re-homing moves a metric and never
removes one, so a derived ${ref} to it still resolves.
Exception: a column named by meta.primary_key or by a join's sql_on is
resolved structurally and Lightdash rejects the whole explore when that
dimension is missing, so there the dimension stays and the metric degrades
to a note. Both branches note, so no metric disappears silently.
Goldens are unchanged: the ecommerce fixture has no colliding names.
The branch was 25 commits behind main and predated the Emit signature change
that threads non-fatal degrade warnings back to the CLI, so it no longer
satisfied the Emitter interface.
- lightdash.Emit now returns ([]string, error). The notes are still ALSO
written into schema.yml as a leading `# semglot:` comment block: that file
is committed and deployed by `lightdash deploy`, so a reader of the
artifact should see what was dropped without going back to the build log.
- README: both sides added a dialect. Kept both rows and spliced the
lightdash column into main's capability matrix rather than choosing one.
- Golden regenerated. The diff is additions only: main's new fixture content
(obt_sales, billing_customer_sk, the dim_customer join) plus one new
collision note, `aov`, which the merged fixture now triggers — the fix
firing on a case main introduced.
go test ./... green, go vet clean, gofmt clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XdPy5mkvNRuGZ59rShLUag
The emitter branch added the target but never touched this file, so the mapping reference described every dialect except lightdash. Adds it to the output table and as a column in the IR-concept mapping, and records its gaps and limits: composite primary keys have nowhere to go (meta.primary_key is a single name), synonyms and enum values fold into the column description, and a Lightdash metric cannot express a filtered aggregate or reference a metric on another model. Also documents a failure mode the `--` notation cannot express. Lightdash puts dimensions and metrics in one namespace per model and resolves a clash in favour of the dimension, silently dropping the metric — so a metric named after a column is not a gap, it is a correctness trap: the layer deploys cleanly and quietly lacks the metric it exists to publish. That is the bug this branch fixes, and the reference should say so rather than leaving the next person to rediscover it from a deploy warning. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XdPy5mkvNRuGZ59rShLUag
"Style" implied an aesthetic preference. This is a compatibility switch, and
its values are literally the YAML key path the meta block is written under
("meta" / "config.meta"), so naming it for the key path makes the name and the
value validate each other: `dbt-meta-key-path: config.meta` needs no comment,
where `meta-style: config.meta` read like a category error.
The profile key moves with it (meta-style -> dbt-meta-key-path).
Also records WHY the option exists rather than only what it does. It cannot be
derived: semglot writes a file that someone else's dbt parses later, so the
emitting process never sees the dbt that will read it. Top-level meta still
parses without warning on dbt 1.11 (verified with --show-all-deprecations
against the real 38-model schema), so "" stays the default; when dbt removes
it, flipping the default to "config.meta" makes the option vestigial and
deletable.
docs/plans/ is left alone: it records what was decided at the time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XdPy5mkvNRuGZ59rShLUag
config.meta has been dbt's preferred form since 1.10 (2025-06-16), with two minors shipped since, and 1.10 deprecated top-level meta. Anyone starting today is on 1.10+, so the legacy form is the one that should need asking for. Only an explicit "meta" now opts back into top-level `meta:`, for a project pinned to dbt 1.9 or earlier. The option stays because it cannot be derived: semglot writes a file that someone else's dbt parses later. This changes emitted output for every caller that did not set the option, hence its own commit. Tests: the content tests read ldDoc's `meta:` shape and assert WHAT is emitted, not where it nests, so emitLightdash pins them to the legacy path rather than having a default change break all of them. Placement is owned by TestLightdashConfigDbtMetaKeyPath, which now builds the emitter directly to check the real default and gained a case for the "meta" opt-out. The golden is generated with default options, so it exercises config.meta end to end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XdPy5mkvNRuGZ59rShLUag
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the
lightdashtarget onto main, and fixes a bug that made it silentlylose metrics when deployed.
The bug
Deploying semglot's generated
schema.ymlto Lightdash Cloud produced:The emitter emitted a metric whose name matched a dimension it also emitted on
the same model. Lightdash resolves the clash in favour of the dimension and
silently drops the metric, so the semantic layer quietly loses the metric it
exists to publish. Downstream this does not look like a semglot bug, it looks
like the analytics agent failing to find a metric.
attributed_revenuewas the emitter manufacturing its own collision: it is nota dimension in the IR at all, the column entry existed only because the emitter
hung a column-level metric on it, and Lightdash turns any
columns[]entryinto a dimension.
The fix
The computed metric is canonical; the colliding column is not emitted as a
dimension. Metrics it hosted are re-homed at model level with an explicit
sql: ${TABLE}.<col>. This matches what the sibling emitters already do —snowflake_semantic_view.gohas aTestSVDedupsDimensionMetricNameCollisionusing
roasonobt_marketing_dailyas its example, anddatabricks_metric_view.goseedsseenwith measure names so the field is theone dropped. Keeping semglot's answer to "roas" identical across targets is the
point of routing everything through one IR.
Exception: a column named by
meta.primary_keyor by a join'ssql_onisresolved structurally and Lightdash rejects the whole explore if it is missing,
so there the dimension stays and the metric degrades to a note. Both branches
emit a note, so nothing is silent either way.
Rejected alternatives: renaming the metric (the name is the downstream
contract), and
meta.dimension.hidden: true(a hidden dimension still occupiesthe name, so the metric is still skipped).
Merged with main
The branch was 25 commits behind and predated the
Emitsignature change thatthreads degrade warnings back to the CLI.
lightdash.Emitnow returns([]string, error); the notes are still also written intoschema.ymlas aleading comment block, since that file is committed and deployed and its reader
should not have to go back to the build log.
README conflicted because both sides added a dialect — kept both, and spliced
the lightdash column into main's capability matrix.
Verified against a live workspace
Not just unit tests. Regenerated over the real 38-model marts source and
redeployed to Lightdash Cloud:
That also settles the one thing that could not be verified from tests alone:
Lightdash accepts the re-homed model-level metric form.
go test ./...green,go vetclean,gofmtclean. Golden diff is additionsonly.