Display the internal database sizes in the batches stats

[Kerollmops]

Related product team resources: N/A
Related product discussion: PRD link with explanations.

Motivation

Sometimes we see databases grow a lot but we don’t have the detailed information on what takes space. We also see long indexing times and the size of the database can explain a lot about this timings.

To alleviate this issue we simply display the size of the databases in the batches and the associated diff after a batch has been indexed..

Usage

Refer to the page of the experimental feature

// This is in the `/batches` - `results.stats.internalDatabaseSizes`.
"internalDatabaseSizes": {
  "main": "1.36 MiB (+1.34 MiB)",
  "external_documents_ids": "944 KiB (+944 KiB)",
  "word_docids": "9.11 MiB (+9.11 MiB)",
  "word_prefix_docids": "3.22 MiB (+3.22 MiB)",
  "exact_word_prefix_docids": "16 KiB (+16 KiB)",
  "word_pair_proximity_docids": "100.69 MiB (+100.69 MiB)",
  "word_position_docids": "17.52 MiB (+17.52 MiB)",
  "word_fid_docids": "10.31 MiB (+10.31 MiB)",
  "word_prefix_position_docids": "9.91 MiB (+9.91 MiB)",
  "word_prefix_fid_docids": "4.47 MiB (+4.47 MiB)",
  "field_id_word_count_docids": "240 KiB (+240 KiB)",
  "documents": "25.41 MiB (+25.41 MiB)"
}

Impacted teams

@meilisearch/docs-team, @meilisearch/integration-team
The new internalDatabaseSizes field is in the batches stats.
⚠️ ⚠️ Note that the content of this object can change at any moment and must not be fixed to certain fields. We recommend mentioning it in the docs and not implementing strict structure in SDKs