Experimental embedding cache

[dureuill]

Feature name	How to enable	Description	Missing for stabilization
embedding cache	You can export the env variable, the command line argument, or the config file to indicate the maximal number of entries in the cache of each distinct embedder. 0 (the default) disables the cache. # You can either use this environment variable (150 cache entries) MEILI_EXPERIMENTAL_EMBEDDING_CACHE_ENTRIES=150 # directly use this CLI argument (150 cache entries) --experimental-embedding-cache-entries=150 # Or via the config file (150 cache entries) experimental_embedding_cache_entries = 150	Enables caching of embeddings produced during search.	Could become an embedder settings, could be removed as we think about the bigger picture of caching queries.

How to use the feature?

• Run Meilisearch with the parameter set as an environment variable or a command line argument flag as described above
• Any text embedding request done at search time will be put in a cache specific to that embedder. Looking for the same text query again will return the previously generated embedding without calling the embedder, as long as the entry remains in cache.
• The total number of cache entries is given by the value of the parameter (0 disables the cache).
• The least recently used entries are evicted first.
• Two embedders sharing the same configuration (source, model, and other determining parameters) share the same cache, even if they were declared in distinct indexes. This is useful for multi-search implementing a "local sharding" strategy.

Why is this feature not stable yet?

• We might want a different API for this parameter
• The usefulness of the cache outside of restricted scenarios like multi-search needs to be assessed
• We may want to remove this feature as it is today as our vision for the bigger picture of search query caching takes shape.

🗣️ You are welcome to give feedback about the score details or ask any question on its usage; we are eager to collect feedback on the feature

[VPetukhov]

Thank you for the feature! Just wanted to note that this is extremely useful in a scenario where users work with filters. Imagine there is the same text query, but we expect a user to play with different inclusion criteria. An example of that would be airbnb-like service. In this case, the same text query can be queried a dozen times by the same user.