v1.24.0 🦞

This release features some improvements with the interaction of the vector store and the searchCutoffMs when using the "vectorStore": "experimental" index setting. It also introduces the metadata header Meili-Include-Metadata on the search request that adds a metadata field to the response. These metadatas contains one uid by query and a reminder of the indexUid and its primary key. We also introduced minor bug fixes around the compaction to improve the interaction with task cancellation.

✨ Enhancement

• Improve the vector store search cutoff by @dureuill in #5945
• Improve compaction behaviors by @Kerollmops in #5946
• Search metadata by @ManyTheFish in #5926

🔩 Miscellaneous

• Adapt the standards of prototypes by @curquiza in #5942
• Bump Dockerfile alpine version to 3.22 by @PedroTroller in #5866

👥 New Contributors

• @PedroTroller made their first contribution in #5866

Full Changelog: v1.23.0...v1.24.0

v1.23.0 🐘

This release introduces a new compact route on the index routes, which appends a new compaction task to the queue. Meilisearch uses an LMDB environment by index, and indexes start to fragment after some time. We have noticed that the indexes generally have 30% fragmentation. By defragmenting the environment, we've seen large (2-4x) speed-ups in terms of search and indexation. This is primarily due to the reordering of the LMDB internal pages and the removal of scattered free pages throughout the file, thereby relocating the content to the beginning.

We also worked on parallelizing the post-processing of facets. We noticed that a lot of time was spent iterating over the prefixes of the index in a single-threaded loop. We redesigned this part of the indexation to make it multi-threaded. We have seen a 4x and 6x improvement in terms of time spent on this operation.

✨ Improvements

• Introduce a task to compact an index by @Kerollmops in #5929
• Parallelize bulk facets & word prefix fid/position docids by @Kerollmops in #5307
• Change Java version in SDK CI by @curquiza in #5910
• Minor improvement in OpenAPI CI by @curquiza in #5834
• Add request uid to search routes by @ManyTheFish in #5863

🦋 Bug Fixes

• Fix ranking score bug when sort is present by @ManyTheFish in #5933
• Synonym performance fix by @ManyTheFish in #5930
• Update README.md to fix newsletter link by @EazyAl in #5911
• Try to fix GH license detection again by @dureuill in #5938

🔩 Miscellenaous

• Remove release-drafter and encourage usage of GitHub-generated notes by @curquiza in #5935
• Show Dependabot dependency upgrade in the changelog by @curquiza in #5900
• Bump actions/setup-go from 5 to 6 by @dependabot[bot] in #5912
• Bump actions/setup-dotnet from 4 to 5 by @dependabot[bot] in #5914
• Bump actions/setup-node from 4 to 5 by @dependabot[bot] in #5915
• Bump sigstore/cosign-installer from 3.9.2 to 3.10.0 by @dependabot[bot] in #5916
• Bump actions/setup-python from 5 to 6 by @dependabot[bot] in #5913

New Contributors

• @EazyAl made their first contribution in #5911

Full Changelog: v1.22.1...v1.23.0

v1.22.3 🐦‍🔥

This version contains a minor fix that affects remote federated search users. If you are not a remote federated search user, it is not necessary to migrate from v1.22.x.

🦋 Bugfixes

• v1.22.2 raised the remote federated search timeout for waiting nodes from 5 to 30s. This version makes it configurable by setting the environment variable MEILI_EXPERIMENTAL_REMOTE_SEARCH_TIMEOUT_SECONDS to a positive integer value. Please note that no CLI flag or configuration entry is available. By @dureuill in #5932

v1.22.2 🐦‍🔥

🦋 Bugfixes

• Remote federated search would declare remotes as "timeout'd" after 5s. This timeout was raised to 30s to accomodate some search configurations. By @dureuill in #5919

v1.22.1

🐛 Bug Fixes

• use the latest version of zerometry that supports collection, lines and multi-lines (#5907) @irevoire
• Delete oldest tasks first (#5906) @dureuill

❤️ Huge thanks to our contributors: @dureuill and @irevoire.

v1.22.0 🐦‍🔥

🚀 Enhancements

• Introduce a new geo backend to store geojson and filter on polygon
  1. Make the _geojson field filterable
  2. Then send your documents with a _geojson field filled with a valid geojson
  3. Filter your documents with the new _geoPolygon filter, or the old _geoBoudingBox and _geoPoints filter

🐛 Bug Fixes

• Document template: Correctly render when indexing first item in array by @dureuill in #5896
• arroy to hannoy conversion fails with binary quantized distances by @nnethercott #5891

❤️ Huge thanks to our contributors: @nnethercott, @Kerollmops, @ManyTheFish, @dureuill and @irevoire.

Full Changelog: v1.21.0...v1.22.0

v1.21.0 🐷

🚀 Enhancements

• Introduce a new vector store backend for better performance, especially if using the binary quantization
  1. Enable the new vectorStoreSetting experimental feature
  2. Then change the vectorSetting index setting to "experimental" for the indexes where you want to try the new vector store
  • Done in #5767 by @Kerollmops
• Add Persian support (update charabia to v0.9.7) (#5848) @ManyTheFish

🐛 Bug Fixes

• Observing the progress trace during indexing no longer removes parts of the trace (#5884) @irevoire
• Fix dumpless upgrade decoding error when upgrading with a rest embedder (#5886) @dureuill.
  • In case you had encountered the issue, use the dumpless upgrade to v1.21 to fix it.

❤️ Huge thanks to our contributors: @ja7ad, @agourlay, @Kerollmops, @ManyTheFish, @dureuill and @irevoire.

v1.20.0 🦟

🚀 Enhancements

• Display the progressTrace in in-progress batches (#5858) @shreeup

🐛 Bug Fixes

• Send the version when returning prometheus metrics (#5876) @irevoire

🔒 Security

• Bump tracing-subscriber from 0.3.19 to 0.3.20 (#5869) @dependabot[bot]

⚙️ Maintenance/misc

• Fix scheduled CI failure (#5856) @arithmeticmean

❤️ Huge thanks to our contributors: @ManyTheFish, @arithmeticmean, @curquiza, @dureuill, @irevoire, @shreeup and dependabot[bot].

v1.19.1 🪸

🐛 Performance improvements

Enhance hybrid search with filter performances

In previous versions of Meilisearch, mixing hybrid search with filters, as shown below, could multiply the search time by hundreds.

{
  "q": "hello world",
  "limit": 100,
  "filter": "tag=science"
  "hybrid": {
    "semanticRatio": 0.5,
    "embedder": "default"
  }
}

Meilisearch will now directly compute the semantic distance with the filtered candidates if only a few candidates come from the filter, instead of searching for the closest embeddings matching the filter in the vector database.

v1.19.0 🪸

🚀 Enhancements

Automatically shard documents to scale horizontally

Meilisearch can now automatically distribute documents between multiple instances using the new sharding feature.

Find a guide on implementing sharding in the documentation.

Note

Sharding is an advanced feature available exclusively in Meilisearch Enterprise Edition (EE).

The EE features are governed by the Business Source License 1.1, which allows you to use, test, and develop with sharding for free in non-production environments. Please reach out to us before using it in production.

Added in #5784 by @dureuill

🐛 Bug Fixes

• Takes the allowed max memory of the container when computing the max memory to use (#5729) @martin-g

❤️ Huge thanks to our contributors: @Kerollmops, @dureuill and @martin-g.