Selective index snapshotting and restoring

[sebastian]

Hi dear folks. I posted on Discord, and discord user Bentxi suggested I post here.

My usecase is the following:

• I have geographically distributed search nodes. I would like to keep uneccessary computational load away from these nodes to keep search snappy for our users
• we have 1000s of indices all of which are semi-small (1k-100k docs, roughly) and are independently updated and updated at different intervals. When an update is needed, I would like it to happen quickly. Most indices are updated once per day or more often

It appears the only way to update the indices is to recreate them on each search node individually which – unfortunately – means I incurr the cost of indexing the documents N times (once per node), rather than just once. It would be amazing if it was possible to snapshot individual indices (or even better extract a compressed version of it through the API) and furthermore upload a snapshot to a running node (without requiring a restart).

Without a feature like this the problem I am currently running into is that the latency between the master data changing and the indices all being updated can get unacceptably high. The cause of this is largely that I serialize index updates to not overload the search nodes as they also have to do the heavy lifting of reindexing the data.

[martin-g]

meilisearch/meilisearch#5670 has been merged recently and will be released with 1.16.0.
Sounds related to what you ask for.

[sebastian]

This is fantastic! Looks much better than what I hoped for (hehe, typical "user asks for Y while really (unknowingly) wanting X" situation).

It isn't clear to me from reading through the PR, but is this more akin to a dump rather than a snapshot in terms of whether the receiving node will need to reindex the documents for the index it's receiving?

[sebastian]

It will be interesting to see the semantics of this in practice:

• what happens if there is a failure transfering the index to another node? Will it retry? How often? Will the task fail "immediately" and reporth a status accordingly?
• what happens if the receiving node already has an index by the same name? Will it be overwritten, or will the transfer fail?

I suppose time will show.

Thanks again, @martin-g

[bpolaszek]

Another question: what about embeddings? If I POST /export from instance A, will instance B get the embeddings along with each document, or will they all be (re)computed by instance B?