[Feature]: RaBitQ1 index with BM25 and no vectors

[qdrddr]

I'd like to benefit from extreme low disk footprint with RaBitQ1 index, but do not store actual fullprecision/rabit4/rabit8 vectors in the DB, and only use the index to get the candidates.

Skipping the full precision semantic search among the candidates.

Once seamntic search is done against the index, I want to search with BM25 to find most similar among the candidates. Basically a hybrid search with BM25 + RaBitQ1.

The goal is extreme low disk footprint and fast search.

This could be very useful for key-value semantic search used with LLM KV cache or prompt caching.

[usamoi]

I'd like to benefit from extreme low disk footprint with RaBitQ1 index, but do not store actual fullprecision/rabit4/rabit8 vectors in the DB, and only use the index to get the candidates.

If you already have a function word2vec that can convert text into vectors, you can get it using CREATE INDEX ON t USING vchordrq ((word2vec(passage_text)::vector(100)) vector_l2_ops). See also Indexes on Expressions. This reduces search performance, but it does indeed avoid storing vectors in the table. You can also enable Rerank in Table at the same, then the vectors would not be stored in any of table or index. But it reduces search performance much more.

Skipping the full precision semantic search among the candidates.

Not sure what you mean. If you're referring to the search process using only 1-bit vectors without involving any full-precision vectors, that's not supported, as it wouldn't achieve reasonable accuracy.

Once seamntic search is done against the index, I want to search with BM25 to find most similar among the candidates. Basically a hybrid search with BM25 + RaBitQ1.

You can achieve this using Common Table Expression and any extension that provides BM25, for example, VectorChord-bm25.

---

cc @VoVAllen

[qdrddr]

Yes, this feature request is to support 1-bit vectors without involving any full-precision vectors. Since the vchordrq index already utilizes rabitq1, we could simply reuse it instead of duplicating it by creating a separate column.

I know rabitq1 wouldn't achieve reasonable accuracy, that's why we can combine it with bm25 to improve the accuracy with Lexical / Keyword Search.

This is intended to specific use-cases like LLM KV Cache or prompt caching where rabitq1 + keyword search could be enough.
For example it could be used with the LiteLLM proxy to cache compressed LLM prompts and reduce costs.

Like this BerriAI/litellm#25650

The goal is to have cheap fast compressed caching that we can use to reduce costs with LLM calls.

[VoVAllen]

@qdrddr Hi, What kind of data scale you're considering? And do you use it for long context or RAG?

[qdrddr]

@qdrddr Hi, What kind of data scale are you considering? And do you use it for long context or RAG?

This is intended for millions of records.

It may be used in an RAG pipeline for some use cases where either accuracy is not much of a concern or where BM25 matching can compensate for 1-bit quantization.

For example, LLM KV cache (for inference engines) or prompt compression caching, such as LiteLLM Proxy.