Skip to content

chore: move the in-development version to 0.3.0.dev0 - #691

Merged
christianeu-db merged 1 commit into
apache:mainfrom
jbonofre:version-0.3.0.dev0
Nov 9, 2026
Merged

christianeu-db merged 1 commit into
apache:mainfrom
jbonofre:version-0.3.0.dev0

Conversation

@jbonofre

@jbonofre jbonofre commented Nov 3, 2026

Copy link
Copy Markdown
Member

Summary

The next release is 0.3.0-incubating, so the in-development version becomes 0.3.0.dev0. There will be no 0.2.0 release.

  • Sets 0.3.0.dev0 everywhere the current version is declared. That covers ossie-schema.json, ontology.json, mapping.json, spec.yaml, the Python package (pyproject.toml, uv.lock, models.py), spec.md, ontology.md, relational_semantics.md, docs/index.md, the examples, the compliance fixtures and the tests.
  • Breaking for documents. The schemas pin the version with const, so documents that declare 0.2.0.dev0 no longer validate. The version history in spec.md says they must declare 0.3.0.dev0, and the ontology tests now reject 0.2.0.dev0. Because it changes the schemas, this counts as a specification change, which needs three binding +1 votes.
  • apache/ossie-converters needs a matching change. It writes 0.2.0.dev0 in about 30 files.
  • One history entry is unchanged. The expression language revision table keeps its dated 0.2.0.dev entry.

The next release is 0.3.0-incubating, so the in-development version
becomes 0.3.0.dev0. There is no 0.2.0 release.

- Set 0.3.0.dev0 in the schemas, spec.yaml, the Python package, the
  specification and ontology documents, the examples, the compliance
  fixtures and the tests.
- Record in the version history that 0.3.0.dev0 replaces 0.2.0.dev0, and
  that documents declaring 0.2.0.dev0 must declare 0.3.0.dev0.
- Reject 0.2.0.dev0 as an embedded model version in the ontology tests.

The expression language revision table keeps its dated 0.2.0.dev entry.

@christianeu-db christianeu-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good - left a few notes to consider but feel free to address them as followups


def _doc(metrics=None, datasets=None):
document = {"version": "0.2.0.dev0", "name": "t"}
document = {"version": "0.3.0.dev0", "name": "t"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker, but no workflow runs anything under compliance/. validation-ci.yml and python-ci.yml do not list it in their paths, and neither runs the grader tests or validates the fixtures. So the green checks on this PR do not cover this file or the nine *.ossie.yaml fixtures. Locally, the 13 grader tests pass and all nine fixtures validate.

This matters more after the rename. #631 adds five more portability fixtures that declare 0.2.0.dev0, and #310 adds an ontology example that no CI step validates. Either one can merge green after this PR and leave a document on main that the schema rejects. Should we add compliance/** to the Validation CI paths, with steps that run the grader tests and validate the fixtures? A follow-up PR is fine too.

Comment thread core-spec/spec.md
### Migrating earlier document shapes

This is a breaking change in the unreleased `0.2.0.dev0` specification. Earlier
This is a breaking change in the unreleased `0.3.0.dev0` specification. Earlier

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the break is still in the 0.2.0.dev0 snapshot. 0.3.0.dev0 should always have the flattening applied.

(non-blocking side note: perhaps these kinds of migration notes should in a separate file rather than the core -spec. the core spec just states the current state w/o reflecting on the past)

@christianeu-db

Copy link
Copy Markdown
Contributor

@jbonofre do you mind updating core-spec/wide_table_semantics.md. This could also be a followup PR (if so, I can take it)

@MonkeyCanCode MonkeyCanCode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@christianeu-db
christianeu-db merged commit c1a3318 into apache:main Nov 9, 2026
15 checks passed
ekut added a commit to Dactopus/dactopus-ossie-clickhouse that referenced this pull request Nov 10, 2026
apache/ossie#691 renamed the in-development version of the schema; there
will be no 0.2.0 release. The vendored apache-ossie moves to d700cde, and
the TPC-DS fixture is again an exact copy of the example on Ossie's main.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
ekut added a commit to Dactopus/dactopus-ossie-clickhouse that referenced this pull request Nov 10, 2026
apache/ossie#691 renamed the in-development version of the schema; there
will be no 0.2.0 release. The vendored apache-ossie moves to d700cde, and
the TPC-DS fixture is again an exact copy of the example on Ossie's main.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
johndacosta added a commit to johndacosta/ossie that referenced this pull request Nov 10, 2026
…filter

Resolved the sole conflict in core-spec/spec.md Version History by keeping
apache#691's 0.3.0.dev0 heading and replacement note, and re-applying apache#704's
Dataset.source table-identifier changelog bullet. Bumped
examples/dataset_sources.yaml from 0.2.0.dev0 to 0.3.0.dev0 to match main.
isocline pushed a commit to xsko/ElastiCORE that referenced this pull request Nov 10, 2026
ekut added a commit to Dactopus/dactopus-data-models that referenced this pull request Nov 11, 2026
apache/ossie#691 moved the spec's version to 0.3.0.dev0; there will be
no 0.2.0. dactopus-ossie-clickhouse 0.15.0 and newer read only the new
version, so CI moves its pin from 0.14.0 to 0.16.0, and AGENTS.md drops
the old name custom_extensions it kept until the pin moved. Nothing else
in the models changes. The DuckDB extension (0.3.0) reads the new
version and answers as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
christianeu-db added a commit that referenced this pull request Nov 11, 2026
PR #691 moved the in-development version to 0.3.0.dev0 everywhere the
version is declared, but the Layer 3 "Wide Table Query Interface" spec
(core-spec/wide_table_semantics.md) kept its draft-spec badge at
0.2.0.dev0. Its Layer 2 sibling (relational_semantics.md) was bumped in
that PR, so this straggler left the layered spec set inconsistent. Bump
the badge to 0.3.0.dev0 to match.

Co-authored-by: Isaac <no-reply@databricks.com>
MrBlack1995 added a commit to MrBlack1995/ossie that referenced this pull request Nov 12, 2026
main moved the in-development version to 0.3.0.dev0 (apache#691), and the
schema pins it, so the fixtures this PR adds declare it too.

Co-authored-by: Isaac <no-reply@databricks.com>
MrBlack1995 added a commit to MrBlack1995/ossie that referenced this pull request Nov 16, 2026
main moved the in-development version to 0.3.0.dev0 (apache#691), and the
schema pins it, so the fixtures this PR adds declare it too.

Co-authored-by: Isaac <no-reply@databricks.com>
jbonofre pushed a commit that referenced this pull request Nov 18, 2026
…form of M3 (#631)

* test(compliance): add portability rung M6, date and time scalars

One dataset with YEAR and DATE_TRUNC fields and an AVG(DATEDIFF(...)) metric,
using the part-first DATEDIFF(part, start, end) order from
expression_language.md. Adds one construct on top of the baseline.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): add the FILTER (WHERE ...) form to portability rung M3

expression_language.md makes the aggregate FILTER (WHERE ...) clause REQUIRED.
Two variants (qualified and bare) express the same filtered aggregate as the
existing CASE WHEN fixtures, so a difference between the two forms isolates
handling of the clause itself.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): add portability rung M4, a cross-dataset metric

Two datasets joined by a declared relationship and one model-scoped metric
that aggregates across the join (SUM over one dataset divided by a COUNT
DISTINCT over the other), referencing fields as dataset.field.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): add portability rung M5, metric-to-metric composition

A model-scoped metric (avg_order_value) composed from two other model-scoped
metrics referenced by name, per Metric Scoping in core-spec/spec.md.

Co-authored-by: Isaac <no-reply@databricks.com>

* docs(compliance): describe portability rungs M0-M6

Lists the M3 FILTER (WHERE ...) form and rungs M4-M6 in the README table,
notes that M7 and M8 wait on the window ORDER BY NULL-placement default, and
records that the fixtures live in this repository.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): cover M4-M6 in the identity test, name every element by kind and name

Follow-ups from the review of #631:

- The identity regression test looped over M0-M3 only; it now covers M0-M6,
  and the grader README says so.
- M5's reference_form_primary said "qualified", but its only variant (the
  composite under test) references metric names; it is now "metric-name".
- M6 named "fields and metric" with a comma-joined name, and M2 named
  "metrics" without a name, so a lookup by kind and name found neither.
  Every rung now lists elements_under_test, each with one kind (metric or
  field) and one name.

Two new tests keep the manifest honest: every listed element is found by
kind and name in each variant of its rung, and each primary variant uses
the rung's primary reference form. Both fail on the previous manifest.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): carry M5's composition inlined, as the spec requires

A metric expression cannot refer to other metrics (core-spec/spec.md,
Metric Scoping; expression_language.md), and since #608 validate.py warns
when one does. M5's composite, avg_order_value = total_amount / order_count,
referenced the two metrics by name. The "Referenced as metric_name" row in
the Metric Scoping table is how a query addresses a metric, not what an
expression may contain.

M5 now carries the composition the way a converter has to emit it when the
source tool composes measures: the parts inlined, over qualified
dataset.field references, SUM(sales.amount) / NULLIF(COUNT(sales.order_id), 0),
NULL when there are no orders. The two base metrics stay alongside it. The
fixture validates with no warnings on main's validator, and its primary
reference form is qualified again.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): write M3's CASE form without ELSE 0, as the FILTER form means

expression_language.md (Conditional Aggregations) states that FILTER (WHERE
...) is equivalent to a CASE with no ELSE for an aggregate that ignores
NULL, and that an ELSE 0 "would count as a real row". M3's CASE fixtures
wrote ELSE 0, so on rows that exist but do not match they give 0 where the
FILTER variants added here give NULL; they were not the same aggregate the
rung describes. Found by idk-arsh's execution check on
apache/ossie-converters#24.

Both CASE fixtures now read SUM(CASE WHEN ... THEN amount END), which also
matches DAX (CALCULATE over no rows is BLANK). All four M3 fixtures
validate on main.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): carry the bare variants as dataset-scoped metrics, grade them

Since #486 a model-scoped metric must reference fields as dataset.field,
and since #608 validate.py warns on the bare variants of M0, M2 and M3
("is not qualified"). Metric Scoping in core-spec/spec.md gives the bare
form its place: a dataset-scoped metric (datasets[].metrics) references its
own dataset's fields by single-part name. The four bare fixtures now carry
their metrics there, and validate without warnings on main.

The grader read only the model-root metrics, so a dataset-scoped metric
was not graded at all, and the identity test still passed. It now collects
datasets[].metrics too, keyed dataset.metric as the spec references them,
and the identity test asserts that every metric in a fixture is graded.
New tests: a dataset-scoped metric graded PORTABLE, dropped, and moved to
the model (REWRITTEN, identity not preserved). All four fail without the
grader change.

Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): name manifest elements by the grader's keys; spec spelling in M6

Follow-ups from the second review of #631:

- The manifest named fields bare (amount_with_tax, order_year,
  order_month_start), but the grader reports a field as dataset.field, so
  matching manifest elements to findings found no result for M1 or M6.
  Fields are now named sales.<field>, and a variant that carries the
  rung's metrics as dataset-scoped metrics says so with metrics_in: sales
  (the grader reports those as sales.<metric>). The manifest test now uses
  the grader's own _collect keys instead of building a name set of its
  own, so the two cannot drift apart; it fails on the previous manifest.
- M6 wrote DATEDIFF('day', ...), the Tableau spelling in the mapping
  table; the spec writes the part as a keyword, DATEDIFF(day, ...).

Co-authored-by: Isaac <no-reply@databricks.com>

* docs(compliance): point the portability ladder at apache/ossie-converters#24

Issue #576 was transferred to apache/ossie-converters#24 with the
converters; the ladder, grader and manifest still cited the old number,
which now resolves to nothing.

Co-authored-by: Isaac <no-reply@databricks.com>

* fix(compliance): keep the dataset qualifier on a field two datasets declare

M4 is the first rung with two datasets, and customer_id is declared by both
sales and customers. The grader stripped every table qualifier before
comparing trees, so a converter that re-pointed COUNT(DISTINCT
customers.customer_id) to sales.customer_id, or dropped the qualifiers, was
graded as a cosmetic rewrite, and _repair_by_expression paired such a
renamed metric as expression-equivalent.

_ambiguous_fields() reads the field names that more than one input dataset
declares; for those the qualifier is kept, for all others it is stripped as
before (M0-M3 have one dataset each, so their grades do not change). Three
regression tests; two fail on the previous grader.

Suggested-by: Jean-Baptiste Onofré <jbonofre@apache.org>
Co-authored-by: Isaac <no-reply@databricks.com>

* test(compliance): move the M3-M6 fixtures to spec version 0.3.0.dev0

main moved the in-development version to 0.3.0.dev0 (#691), and the
schema pins it, so the fixtures this PR adds declare it too.

Co-authored-by: Isaac <no-reply@databricks.com>

* fix(compliance): resolve a dataset qualifier against the dataset that declares the field

Review of #631. The shared-field rule kept or dropped a qualifier by the
field name alone, so a field requalified with its own dataset needed
review when another dataset declared the name too, and a metric that
named a dataset not declaring the field was cosmetic. Each column is now
resolved to the dataset it names before the trees are compared: a bare
name in a field expression or dataset-scoped metric is the element's own
dataset, elsewhere the one dataset that declares it, and a written
qualifier is kept as written.

Co-authored-by: Isaac <no-reply@databricks.com>

* fix(compliance): treat a returned dataset with an authored dataset's source as that dataset

A converter that names a dataset after its source table (lightdash returns
orders for an authored sales over samples.sales.orders) renamed it. Since
qualifiers are kept as written, its metrics and fields were graded as dropped
plus an unrelated addition. Map the renamed dataset's qualifiers on the
returned side to the authored dataset before resolving, when exactly one
authored dataset has that source. The 120 cases of the latest Interop matrix
run now grade as on main. Adds a test built from lightdash's real M0 return.

Co-authored-by: Isaac <no-reply@databricks.com>

* docs(compliance): M5 carries composition inlined, without claiming metrics cannot reference metrics

Since #703 a model-scoped metric may evaluate a dataset-scoped metric with
AGG(dataset.metric). Reword the M5 fixture comment, the manifest construct and
the README row.

Co-authored-by: Isaac <no-reply@databricks.com>

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants