Releases: opendatacube/datacube-core
Open Data Cube release 1.9.7
This release includes a fix for a critical bug affecting dask loading and the postgis index driver.
What's Changed
- Fix spell check by @pjonsson in #2033
- eo3: fix type error by @pjonsson in #2031
- cog: un-nest and align logic by @pjonsson in #2034
- Enable flake8-simplify rules by @pjonsson in #2035
- search_tool: fix some types by @pjonsson in #2036
- Fix type checker errors by @pjonsson in #2037
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2043
- Handle list type resolution by @Ariana-B in #2039
- Fix type checker errors by @pjonsson in #2042
- validate_document: add type annotations by @pjonsson in #2041
- BandInfo: deprecate unused 3D parameter by @pjonsson in #2040
- Restrict to sphinx-tabs 3.4.5 by @pjonsson in #2046
- docs: add missing index by @pjonsson in #2047
- tests: test legacy resolution by @pjonsson in #2050
- Dockerfile: set UV_LINK_MODE by @pjonsson in #2051
- CI: pin image by hash by @pjonsson in #2048
- Relax click requirement by @pjonsson in #2052
- Extend and integrate
native_load
by @emmaai in #2053 - Enable more lint rules by @pjonsson in #2056
- Revert "Restrict to sphinx-tabs 3.4.5" by @pjonsson in #2057
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2063
- Fix type checker errors by @pjonsson in #2062
- xarray_geoextensions: add space in warning by @pjonsson in #2060
- CI: check right file in pre-commit by @pjonsson in #2064
- pylint: remove reports parameter by @pjonsson in #2061
- Require setuptools_scm >= 8 by @pjonsson in #2065
- testutils: fix NoData behavior by @pjonsson in #2049
- CI: shorten job timeout by @pjonsson in #2066
- readthedocs: remove extra_requirements by @pjonsson in #2067
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2070
- build(deps): bump github/codeql-action from 3.29.2 to 3.29.4 by @dependabot[bot] in #2072
- build(deps): bump astral-sh/setup-uv from 6.3.1 to 6.4.3 by @dependabot[bot] in #2073
- Typecheck tests in CI by @pjonsson in #2071
- CI: run code coverage earlier by @pjonsson in #2076
- docs: introduce docs/source directory by @pjonsson in #2068
- docs: build with uv by @pjonsson in #2082
- create_storage: change parameter type by @pjonsson in #2078
- Various type signature adjustments by @pjonsson in #2079
- dockerignore: update with more patterns by @pjonsson in #2084
- docs: autogenerate version by @pjonsson in #2085
- pre-commit: update ruff hook name by @pjonsson in #2086
- CI: use OIDC for CodeCov by @pjonsson in #2088
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2089
- Ensure MetadataType models can always be pickled to support Dask loading by @SpacemanPaul in #2083
- Make configure_s3_access driver aware by @SpacemanPaul in #2087
Full Changelog: 1.9.6...1.9.7
Includes contributions from @pjonsson, @SpacemanPaul, @emmaai and @Ariana-B
The Open Data Cube Steering Council thanks and acknowledges the ongoing support of Geoscience Australia and RISE.
Open Data Cube 1.9.6
This release brings odc-stac
in as a direct dependency of datacube-core
and moves some STAC to EO3
conversion code out of odc-stac into core as the first step in an organisation-wide cross-repo consolidation and
refactor of STAC handling code and in preparation for more native suppport of STAC metadata within core.
It also contains some enhancements to CLI commands, bugfixes and cleanup.
What's Changed
- build(deps): bump the python-deps group with 15 updates by @dependabot[bot] in #1969
- examples: fix function signatures by @pjonsson in #1972
- examples: remove unimplemented zeros by @pjonsson in #1974
- docs: remove broken link by @pjonsson in #1973
- examples/tests: fix dependencies by @pjonsson in #1975
- docs: escape trailing underscores by @pjonsson in #1976
- docs: miscellaneous small fixes by @pjonsson in #1981
- docs: fix links in migration document by @pjonsson in #1980
- Add dataset count CLI command by @Ariana-B in #1963
- Dockerfile: use uv for installation by @pjonsson in #1971
- docs: add missing links by @pjonsson in #1978
- PR and release process updates by @omad in #1983
- docs: stop redefining label by @pjonsson in #1984
- docs: make Sphinx resolve ODCEnvironment by @pjonsson in #1985
- Put JSON type aliases in separate file by @pjonsson in #1986
- docs: add explicit code-block by @pjonsson in #1988
- Remove unused PY35 constant by @pjonsson in #1987
- docs: document named tuple parameters by @pjonsson in #1977
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #1990
- CI: run with fewer permissions by @pjonsson in #1997
- drivers: make types more specific by @pjonsson in #1996
- Update dependencies by @pjonsson in #1994
- Various CLI improvements by @Ariana-B in #1993
- cli: only read files once by @pjonsson in #2000
- Update to mypy 1.16.1 by @pjonsson in #1995
- tests: fix return value checks by @pjonsson in #2001
- Switch decorator order to appease lint/typing by @omad in #2004
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2007
- Type-annotate type aliases by @pjonsson in #2006
- build(deps): bump igsekor/pyspelling-any from 1.0.4 to 1.0.5 by @dependabot[bot] in #2008
- Use datetime.UTC instead of datetime.pytz() by @omad in #2010
- Avoid Null Reference in Core by @omad in #2011
- core: remove old assert by @pjonsson in #2013
- CI: test with Python 3.13 by @pjonsson in #2015
- CI: pin actions by hash by @pjonsson in #2014
- Record rasterio/gdal/etc versions from CI runs by @omad in #2017
- Depend on odc-loader by @pjonsson in #2019
- Fix some type signatures by @pjonsson in #2021
- postgis: pass plugins parameter by @pjonsson in #2022
- Normalise geobox inputs to Datacube api by @Ariana-B in #2018
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #2023
- build(deps): bump aquasecurity/trivy-action from 0.31.0 to 0.32.0 by @dependabot[bot] in #2025
- build(deps): bump conda-incubator/setup-miniconda from 3.1.1 to 3.2.0 by @dependabot[bot] in #2026
- Depend on odc-stac, and move odc.stac.eo3 from odc-stac to datacube.metadata by @SpacemanPaul in #2027
- Update fallback version number and whats_new.rst for 1.9.6 release by @SpacemanPaul
Full Changelog: 1.9.5...1.9.6
Includes contributions from @pjonsson, @Ariana-B, @omad, and @SpacemanPaul.
The Open Data Cube Steering Council thanks and acknowledges the ongoing support of
Geoscience Australia, CSIRO and RISE.
Open Data Cube 1.9.5
This release introduces support for boolean type search fields, as well as a handful of bug-fixes.
What's Changed
New feature:
Bug fixes:
- Export grid workflow classes from datacube.api fixing issue #1953 #1958
archive-less-mature
runs when re-indexing datasets (fixing issue #1917) #1948- Serialise Measurement with canonical name (fixing issue #1936) #1962
Docker image improvements:
Miscellaneous code cleanup and CI improvements:
- model: handle integer resolution #1967
- Remove types-redis dependency #1965
- index: add return types #1939
- Update dependencies #1925
- Run Ruff format on all code #1926, #1927
- Group all dependabot updates in the 'uv' ecosystem. #1928, #1934
- Move deprecated imports #1938
- Remove empty doc strings #1946
- index: fix search_by_product type #1944
- Put coverage config in pyproject #1942
- Fix alembic deprecation warning #1941
- Convert doc strings to type annotations #1940, #1950, #1951
- Convert doc strings to type annotations #1940
- Document alembic upgrades #1955
- postgis: stop parsing table names #1961
- postgis: fix alembic metadata object #1960
- increment fallback version and finalise release notes #1970
Automated updates
Includes contributions from @pjonsson, @omad, @emmaai, @Ariana-B and @SpacemanPaul, with thanks to
supporting organisations CSIRO, RISE and Geoscience Australia.
Full Changelog: 1.9.4...1.9.5
v1.9.4 (20 May 2025)
Improvements
Bring back GridWorkflow
GridWorkflow is back! It was accidentally removed with the deprecated ingestion tools as part of the ODC 1.9 release, since it’s no longer needed internally. It is however an extremely useful public API for anyone running large scale data summaries and is used by ODC Statistician.
GridWorkflow has been updated to use the odc-geo geometry classes.
See: issue 1749 and PR 1760
Fixed Dask load crash in some cases
There was a regression in data loading with dask in Datacube 1.9. It only occurred in rare cases so only affected some people.
The load failed whenever there were completely empty data chunks, which are rectangular in shape. That is, X != Y.
See PR 1780 and issue 1779 for details.
Cleaning and Tidying
Massive cleaning effort across the codebase, spearheaded by @pjonsson, with a focus on:
Improvements in type annotations
Linting fixes
Code formatting and import ordering
Continuous integration improvements
See the ODC Docs Whats New page for more details.
Other Changes
- Update whats_new.rst for 1.9.3 release. by @SpacemanPaul in #1778
- "Opt-in" use of odc-geo GridSpec by @Ariana-B in #1783
- sql: fix packagenames type by @pjonsson in #1784
- Add some type signatures by @pjonsson in #1785
- tests: use set instead of list by @pjonsson in #1792
- users: import right index by @pjonsson in #1793
- Remove executable permission on files by @pjonsson in #1797
- Fix Dask Data Load error by @omad in #1780
- Use Ruff instead of flake8 by @pjonsson in #1789
- dependabot: update uv.lock by @pjonsson in #1790
- postgres: use one type for fields by @pjonsson in #1795
- Import names from defining module by @pjonsson in #1799
- build(deps): bump pytest-cov from 6.0.0 to 6.1.1 by @dependabot in #1807
- build(deps): bump urllib3 from 2.3.0 to 2.4.0 by @dependabot in #1806
- build(deps): bump ipython from 8.34.0 to 8.35.0 by @dependabot in #1804
- build(deps): bump beautifulsoup4 from 4.13.3 to 4.13.4 by @dependabot in #1803
- build(deps): bump mako from 1.3.9 to 1.3.10 by @dependabot in #1805
- dependabot: group uv dependencies by @pjonsson in #1816
- build(deps): bump jsonschema-specifications from 2024.10.1 to 2025.4.1 by @dependabot in #1814
- build(deps): bump astral-sh/setup-uv from 5 to 6 by @dependabot in #1820
- postgis: inherit instead of decorate by @pjonsson in #1801
- fields: type annotate offset parameter by @pjonsson in #1809
- virtual: fix pydoc formatting by @pjonsson in #1810
- tests: update to moto 5.1.4 by @pjonsson in #1822
- CI: use the right project slug by @pjonsson in #1823
- build(deps): bump the uv-deps group with 28 updates by @dependabot in #1821
- Various lint fixes by @pjonsson in #1824
- geometry: stop misusing constructor by @pjonsson in #1827
- tests: fix midnight bug by @pjonsson in #1825
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1832
- build(deps): bump the uv-deps group across 1 directory with 3 updates by @dependabot in #1834
- build(deps): bump the uv-deps group with 2 updates by @dependabot in #1840
- api: type annotate group_by_time by @pjonsson in #1788
- postgis: update mapper dynamically by @pjonsson in #1831
- CI: add security scanner by @pjonsson in #1833
- dockerfile: stop running chown by @pjonsson in #1835
- dependabot: update boto* weekly instead by @pjonsson in #1837
- Python 3.10 updates by @pjonsson in #1839
- More lint fixes by @pjonsson in #1838
- Add some more type signatures by @pjonsson in #1796
- build(deps): bump the uv-deps group with 3 updates by @dependabot in #1841
- pytest: configure in pyproject.toml by @pjonsson in #1844
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1845
- Dependabot: fix boto group conflict by @pjonsson in #1843
- Fix file permissions by @pjonsson in #1846
- Dockerfile: use uv-generated constraints by @pjonsson in #1842
- build(deps): bump the uv-deps group with 5 updates by @dependabot in #1853
- Sort imports with Ruff by @pjonsson in #1858
- Fix docstring parameter names by @pjonsson in #1857
- dependabot: add boto-stubs to weekly by @pjonsson in #1856
- Add types for pandas while developing by @omad in #1867
- Dockerfile: add panda types by @pjonsson in #1868
- build(deps): bump the uv-weekly-deps group across 1 directory with 4 updates by @dependabot in #1860
- build(deps): bump hypothesis from 6.131.9 to 6.131.12 in the uv-deps group by @dependabot in #1861
- build(deps): bump hypothesis from 6.131.9 to 6.131.12 by @dependabot in #1859
- Speed up S3 Client tests with mocks by @omad in #1878
- Fix more docstrings by @pjonsson in #1880
- Reinstate grid workflow by @omad in #1760
- pyproject: set a fallback version by @pjonsson in #1887
- Set a valid fallback version by @pjonsson in #1889
- Dependabot: disable weekly updates by @pjonsson in #1886
- We don't use compliance-checker any more by @omad in #1888
- Remove cf-units leftovers by @pjonsson in #1897
- build(deps): bump the uv-deps group across 1 directory with 5 updates by @dependabot in #1895
- model: remove old asserts by @pjonsson in #1898
- Enable flake8-comprehensions by @pjonsson in #1900
- Remove pytest-timeout by @pjonsson in #1899
- dependabot: set to weekly schedule by @pjonsson in #1901
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1902
- Ensure lock files are updated using pre-commit by @omad in #1890
- CI: run tests with docker compose by @pjonsson in #1896
- build(deps): bump botocore-stubs from 1.38.10 to 1.38.14 by @dependabot in #1904
- build(deps): bump boto3-stubs from 1.38.10 to 1.38.14 by @dependabot in #1905
- build(deps): bump the uv-deps group with 3 updates by @dependabot in #1906
- build(deps): bump botocore from 1.38.10 to 1.38.14 by @dependabot in https://github.com/opendatacube/datac...
1.9.3
Contains a fix to a serious bug affecting dask usage, preparation for psycopg3, and a lot of other minor cleanup,
tweaks and fixes.
Includes contributions from @pjonsson, @omad, @caitlinadams, @SpacemanPaul and first PRs from new contributors
@emmanuel-ferdman and @allrob23.
Special thanks to @pjonsson who is single-handedly responsible for most of the PRs in this release, and to
to all supporting organisations including Geoscience Australia, CSIRO and RISE.
Known Issue
There are still some issues with Dask loading in 1.9. We are looking into them and anticipate a 1.9.4 release
to fix them in the next few weeks.
Full list of changes:
Bug Fixes
Warnings Cleanup:
- Fix some readthedocs warnings #1762
- config.py: emit ODC2DeprecationWarning #1733
- tests: ignore deliberate warnings #1741, #1754, #1768
- Add a common table expression to fix a SQLAlchemy 2.0 sub-query warning #1747
- Fix ODC2DeprecationWarnings #1737, #1738, #1745
- Undeprecate Product.grid_spec #1770
Minor Fixes and Cleanup:
- Replace setup.py with pyproject.toml #1670
- Documentation fixes and tweaks #1722, #1725
- Update dependencies #1727, #1731, #1750
- Convert examples and tests to pyproject.toml #1748
- Update to Python 3.10 syntax #1756, #1757, #1758, #1759
- Preparations for Psycopg3 support #1763, #1764, #1765, #1766
- Optimize index_drivers() to return a set #1774
- Various minor cleanups #1772
- api: add missing staticmethod #1773
- whats_new.rst updates #1769, #1778
Typechecking Cleanups:
- Type check cleanups #1744, #1746, #1742, #1755
- Remove redundant casts #1758
- Add override annotations #1767
CI Fixes and Improvements:
- CI: fix docker image name #1730
- CI: fix codecov warning #1740
- CI: retry more in setup-miniconda #1753
- CI: cancel old PR jobs #1761
- CI: update doctor rst version #1775
- Stop building Python 2 wheels #1752
- pytest: only test documentation for relevant files #1751
- Autoupdates #1723, #1726, #1729, #1771
- Update mypy version #1743
1.9.2
What's Changed
- Fix postgis filtered count bug (#1717)
- Suppress cartesian join warnings (#1719)
- Remove stray debug message and minor mypy cleanup (#1720)
- Update whats_new ready for 1.9.2 release (#1720)
Full Changelog: 1.9.1...1.9.2
The work in this release was funded by Auspatious for the Digital Earth Pacific project.
Includes contributions from @SpacemanPaul
1.9.1
Changes
- Make compatible with Dask/Distributed versions 2024.11 and above. A change in how Dask transmitted ODC data structures broke data loading in these versions. We had a dependency restriction to prevent incompatible versions being installed, which is now removed. PR 1706
- New Documentation Theme which scales properly for mobiles, includes a dark mode, looks cleaner and is much more maintainable. PR 1676, PR 1702
- Extend the allowed range of PostgreSQL TCP Ports to 65,535. Known as dynamic or ephemeral port numbers, they aren’t used for permanent servers, but are useful when dynamically running servers for tests. PR 1697
Test fixes
- Fix compatibility with antimeridian 0.4+, which includes a superior implementation of splitting, using great circle coordinates instead of a simple 2D plane. PR 1708
- Make tests pass when run as root, which often happens within a docker container. PR 1705
- Hard code the use of nanosecond precision within xarray data. PR 1707
Maintenance
1.9.1rc2
What's Changed
- [chore] Update pre commit hooks, copyright dates for 2025, Doc-QA checks, mypy version pin by @omad in #1699
- Dra/extend valid db port range by @omad in #1697
- Major documentation theme update and fixes by @omad in #1676
- Fix: Use canonical RTD url everywhere by @omad in #1702
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1701
- Fix dask 2024.11+ incompatibility by @omad in #1706
- Hardcode to ns precision times in xarray by @omad in #1707
- Fix failing configuration test when running as root (and fix potential crash on misconfigured config paths) by @omad in #1705
- Fix tests for antimeridian 0.4+ by @omad in #1708
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1710
- In GHA Always Build Test Docker (And don't rely on DockerHub/credentials) by @omad in #1711
- Improve automated release process with OIDC and uv by @omad in #1713
- Prepare release notes for 1.9.1 release by @omad in #1714
- Allow GHA to Authenticate with OIDC to PyPI by @omad in #1715
Full Changelog: 1.9.0...1.9.1rc2
Open Data Cube 1.9.0
Open Data Cube 1.9.0 release
1.9.0 is first major release of the Open Data Cube in several years. The focus has been on retiring technical
debt and unused features and preparing for major architectural changes in subsequent releases.
As there are some backwards incompatibilities in this release, we recommend that you read the migration notes
at docs/installation/MIGRATION-1.8-to-1.9
before upgrading.
Major changes from recent 1.8.x releases include:
-
A new index driver that uses PostGIS spatial indexes to provide faster and more accurate geospatial search
and provide better support for storing data that covers regions where traditional lat/long seach is
inadequate (i.e. covering polar regions or crossing the anti-meridian).The new postgis index driver implements a new API for working with dataset lineage, and only supports
a single location per dataset. It is otherwise largely backwards compatible with the legacy postgres index driver.The legacy "postgres" index driver is still available, but will be removed in a future release.
-
A new configuration layer that provides more predictable and consistent behaviour and will be easier to
extend in future.Given the nature and scope of the changes, the new configuration layer is not fully backwards compatible
with the old implementation, but the vast majority of use cases should only require minor adjustments
to existing configuration and code at most. -
The old
datacube.utils.geometry
library is now deprecated.odc-geo
is used internally throughout,
and we recommend updating all your code to use odc-geo rather than the deprecated internal library. -
The long-deprecated executor and ingestion workflows have been removed.
Changes since 1.9.0-rc13
- API autodocs cleanup (#1688)
- Further metadata fix for new lineage API (#1690)
- Update release process ready for post-1.9.0 release (#1691)
- Removed all references to the postgis driver as "experimental" in tests and documentation (#1693)
- Update whats_new.rst etc. for 1.9.0 release (#1694)
With thanks to all who contributed to 1.9.0 development: @omad, @whatnick, @pjonsson, @alexgleith, @Kirill888, @robbibt, @snowman2, @caitlinadams, @Ariana-B, and @SpacemanPaul
Open Data Cube 1.9.0-rc13 pre-release
Open Data Cube 1.9.0-rc13
Pre-release of the lineage API bug-fix needed for Explorer pre-release.
This will most likely be the last release candidate before the 1.9.0 release.
Changes since 1.9.0-rc13
- Update and cross-reference 1.8 to 1.9 migration notes (#1686)
- Fix SimpleDocNav lineage handling with PostGIS index (#1687)
- Update whats_new.rst for 1.9.0-r13 pre-release (#1689)
Includes contributions from @SpacemanPaul and @Ariana-B