Skip to content
This repository was archived by the owner on Mar 21, 2024. It is now read-only.

Commit 2b7cc90

Browse files
gmourierirevoireKerollmopsdureuillbidoubiwa
authored
Release v1.1.0 (#220)
* Bump API version * Metrics API — Introduce a `/metrics` route for observability / monitoring purposes (#174) * init spec for /metrics * Rename spec filename and add description for metrics * update metrics name and type col in the metrics table; add auth errors * Add --enable-metrics-route to instance-options.md specification * Precise metrics * update open-api.yml * Add telemetry instance property * Mention metrics route on stats.get action * Add action * Precise action description * Rephrase future possibility * Add MEILI_ENABLE_METRICS_ROUTE * rename all the cli parameter to the experimental variant * Update text/0174-metrics-api.md Co-authored-by: Guillaume Mourier <[email protected]> * Update open-api.yaml Co-authored-by: Guillaume Mourier <[email protected]> * Update text/0085-api-keys.md Co-authored-by: Guillaume Mourier <[email protected]> * Apply suggestions from code review Co-authored-by: Tamo <[email protected]> --------- Co-authored-by: Tamo <[email protected]> * API Keys - Introduce `*` wildcard char at the last position to express access to multiple indexes starting with the same string (#189) * Bump open-api spec * Precise that the * wildcard char can be used at the last position to match multiple indexes starting with the same string * Update future possibilities * Update the index wildcard pattern error message --------- Co-authored-by: Clément Renault <[email protected]> * Documents API — Support a csv delimiter customization (#221) * specify the support of the csv-delimiter * Apply suggestions from code review * get rid of the last 'future possibility' that is being described in this PR * Update openAPI specification --------- Co-authored-by: Guillaume Mourier <[email protected]> * Update the geo-search spec to mention the fact that null is a valid geo field (#222) * GeoSearch — Add the `_geoBoundingBox` built-in filter (#223) * Add geoBoundingBox built-in filter description * Add missing parts * Update text/0059-geo-search.md Co-authored-by: Tamo <[email protected]> * Replace the top left / bottom right signature to a top right / bottom left signature --------- Co-authored-by: Tamo <[email protected]> * Search API — Add `facetStats` description (#224) * update search-api.md * rephrase * update open-api.yml * rephrase, add example on open-api.yml * Add examples * fix * Update text/0118-search-api.md Co-authored-by: Louis Dureuil <[email protected]> * rephrase * Clarify that facet stats are for all search results --------- Co-authored-by: Louis Dureuil <[email protected]> * Multi Search API (#225) * Add multi-search spec * Change possible HTTP return codes for index_not_found * Add multi-search to openAPI * Update telemetry * openAPI: shorter summary for multi-search * openAPI: attempt to remove superfluous "values" field in multisearch request/response * Apply review suggestions for openAPI Co-authored-by: Guillaume Mourier <[email protected]> * Apply review suggestions for multi search API Co-authored-by: Guillaume Mourier <[email protected]> * Add queries/results keys to openAPI examples Co-authored-by: Guillaume Mourier <[email protected]> * fix indent * Apply review suggestions for multi search API Co-authored-by: cvermand <[email protected]> * OpenAPI: Change summary to `Multi Search` * use backticks instead of ticks in multi search API example * Apply review suggestions for multi search API Co-authored-by: Maryam <[email protected]> * remove superfluous newline from telemetry file Co-authored-by: Guillaume Mourier <[email protected]> --------- Co-authored-by: Guillaume Mourier <[email protected]> Co-authored-by: cvermand <[email protected]> Co-authored-by: Maryam <[email protected]> * Add small blurb on index count / max size (#226) --------- Co-authored-by: Tamo <[email protected]> Co-authored-by: Clément Renault <[email protected]> Co-authored-by: Louis Dureuil <[email protected]> Co-authored-by: cvermand <[email protected]> Co-authored-by: Maryam <[email protected]>
1 parent aab0b02 commit 2b7cc90

11 files changed

+704
-146
lines changed

‎open-api.yaml

Lines changed: 326 additions & 108 deletions
Large diffs are not rendered by default.

‎text/0028-indexing-csv.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Also, in order to boost write performance CSV data format is more suited than JS
1919

2020
- The header of the csv payload allows to name the attributes and type them.
2121
- `text/csv` Content-Type header is now supported.
22+
- A new query parameter, `csvDelimiter`, has been introduced to customize the csv delimiter used in the document. It can change between two `documentAddition`.
2223
- The error cases have been strengthened and completed. See Errors part.
2324

2425
### II. Motivation
@@ -43,11 +44,15 @@ While there's [RFC 4180](https://tools.ietf.org/html/rfc4180) as a try to add a
4344
4445
- The following CSV lines will represent a document for Meilisearch.
4546
- A `,` character must separate each cell.
46-
- A CSV value should be enclosed in double-quotes when it contains a comma character or a newline to escape it.
47+
- A CSV value should be enclosed in double-quotes when it contains the delimiter character or a newline to escape it.
4748
- Using double-quotes to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote as mentioned in [RFC 4180](https://tools.ietf.org/html/rfc4180).
4849
- Float value should be written with a `.` character, like `3.14`.
4950
- CSV text should be encoded in UTF8.
5051
- The format can't handle array cell values. We are providing `nd-json` format to deal with theses types of attribute in a easier way.
52+
- A `csvDelimiter` query parameter is available to customize the delimiter used in the documents.
53+
- This `csvDelimiter` is optional. By default, the `,` character is used.
54+
- The separator must be one [ascii char](https://www.rfc-editor.org/rfc/rfc20).
55+
- The separator can't be used with another Content-Type, or else it'll throw an error.
5156

5257
##### `null` value
5358

@@ -166,6 +171,7 @@ curl \
166171
- 🔴 Sending a payload excessing the limit will lead to a 413 Payload Too Large - **payload_too_large** error code.
167172
- 🔴 Sending an invalid CSV format will lead to a 400 bad_request - **malformed_payload** error code.
168173
- 🔴 Sending a CSV header that does not conform to the specification will lead to a 400 bad_request - **malformed_payload** error code.
174+
- 🔴 Sending an invalid csv delimiter: not exactly one ASCII char. This will lead to a 400 bad_request - **invalid_document_csv_delimiter** error code.
169175

170176
##### Errors Definition
171177

@@ -267,4 +273,3 @@ n/a
267273

268274
- Provide an interface in the future dashboard to upload CSV data into an index and optionally provide the headers types.
269275
- Set a payload limit directly related to the type of data format. Currently, the payload size is equivalent to [JSON payload size](https://docs.meilisearch.com/reference/features/configuration.html#payload-limit-size). Metrics on feature usage and configuration update should help to choose a better suited value for this type of data format.
270-
- Accepts more common CSV separators

‎text/0034-telemetry-policies.md

Lines changed: 31 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
4040
| Launched | Occurs when MeiliSearch is launched the first time. |
4141
| Documents Searched POST | Aggregated event on all received requests via the `POST` - `indexes/:indexUid/search` route during one hour or until a batch size reaches `500Kb`. |
4242
| Documents Searched GET | Aggregated event on all received requests via the `GET` - `/indexes/:indexUid/search` route during one hour or until a batch size reaches `500Kb`. |
43+
| Documents Searched by Multi-Search POST | Aggregated event on all received requests via the `POST`- `/multi-search` route during one hour or until a batch size reaches `500Kb`. |
4344
| Documents Added | Aggregated event on all received requests via the `POST` - `/indexes/:indexUid/documents` route during one hour or until a batch size reaches `500Kb`. |
4445
| Documents Updated | Aggregated event on all received requests via the `PUT` - `/indexes/:indexUid/documents` route during one hour or until a batch size reaches `500Kb`. |
4546
| Documents Deleted | Aggregated event on all received requests via the `DELETE` - `/indexes/:indexUid/documents`, `DELETE` - `/indexes/:indexUid/documents/:documentId`, `POST` - `indexes/:indexUid/documents/delete-batch` routes during one hour or until a batch size reaches `500Kb`. |
@@ -97,6 +98,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
9798
| `infos.ssl_resumption` | `true` if `--ssl-resumption`/`MEILI_SSL_RESUMPTION` is specified, otherwise `false` | false | Every Hour |
9899
| `infos.ssl_tickets` | `true` if `--ssl-tickets`/`MEILI_SSL_TICKETS` is specified, otherwise `false` | false | Every Hour |
99100
| `infos.with_configuration_file` | `true` if the instance is launched with a configuration file, otherwise `false` | false | Every Hour |
101+
| `infos.experimental_enable_metrics` | `true` if `--experimental-enable-metrics`/`MEILI_EXPERIMENTAL_ENABLE_METRICS` is specified at launch, otherwise `false` | `false` | Every Hour |
100102
| `system.distribution` | Distribution on which MeiliSearch is launched | Arch Linux | Every hour |
101103
| `system.kernel_version` | Kernel version on which MeiliSearch is launched | 5.14.10 | Every hour |
102104
| `system.cores` | Number of cores | 24 | Every hour |
@@ -106,14 +108,15 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
106108
| `stats.database_size` | Database size. Expressed in `Bytes` | 2621440 | Every hour |
107109
| `stats.indexes_number` | Number of indexes | 2 | Every hour |
108110
| `start_since_days` | Number of days since instance was launched | 365 | Every hour |
109-
| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen` |
111+
| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST` |
110112
| `requests.99th_response_time` | Highest latency from among the fastest 99% of successful search requests | 57ms | `Documents Searched POST`, `Documents Searched GET`|
111-
| `requests.total_succeeded` | Total number of successful requests in this batch | 3456 | `Documents Searched POST`, `Documents Searched GET` |
112-
| `requests.total_failed` | Total number of failed requests in this batch | 24 | `Documents Searched POST`, `Documents Searched GET` |
113-
| `requests.total_received` | Total number of received requests in this batch | 3480 | `Documents Searched POST`, `Documents Searched GET`, `Documents Deleted`, `Health Seen`, `Tasks Seen` |
113+
| `requests.total_succeeded` | Total number of successful requests in this batch | 3456 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` |
114+
| `requests.total_failed` | Total number of failed requests in this batch | 24 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` |
115+
| `requests.total_received` | Total number of received requests in this batch | 3480 | `Documents Searched POST`, `Documents Searched GET`, `Documents Deleted`, `Health Seen`, `Tasks Seen`, `Documents Searched by Multi-Search POST` |
114116
| `sort.with_geoPoint` | `true` if the sort rule `_geoPoint` was used in this batch, otherwise `false` | true | `Documents Searched POST`, `Documents Searched GET` |
115117
| `sort.avg_criteria_number` | Average number of sort criteria among all requests containing the `sort` parameter in this batch | 2 | `Documents Searched POST`, `Documents Searched GET` |
116118
| `filter.with_geoRadius` | `true` if the filter rule `_geoRadius` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
119+
| `filter.with_geoBoundingBox` | `true` if the filter rule `_geoBoundingBox` was used in this batch, otherwise `false`| false | `Documents Searched POST`, `Documents Searched GET` |
117120
| `filter.most_used_syntax` | Most used filter syntax among all requests containing the `filter` parameter in this batch | string | `Documents Searched POST`, `Documents Searched GET` |
118121
| `q.max_terms_number` | Highest number of terms given for the `q` parameter in this batch | 5 | `Documents Searched POST`, `Documents Searched GET` |
119122
| `pagination.max_limit` | Highest value given for the `limit` parameter in this batch | 60 | `Documents Searched POST`, `Documents Searched GET` |
@@ -169,6 +172,11 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
169172
| `filtered_by_before_finished_at` | `true` if tasks are filtered by the `beforeFinishedAt` query parameter, otherwise `false` | false | `Tasks Seen`, `Tasks Canceled`, `Tasks Deleted` |
170173
| `filtered_by_after_finished_at` | `true` if tasks are filtered by the `afterFinishedAt` query parameter, otherwise `false` | false | `Tasks Seen`, `Tasks Canceled`, `Tasks Deleted` |
171174
| `per_index_uid` | `true` if an uid is used to fetch an index stat resource, otherwise `false` | false | `Stats Seen` |
175+
| searches.avg_search_count | The average number of search queries received per call for the aggregated event. | `4.2` | `Documents Searched by Multi-Search POST` |
176+
| searches.total_search_count | The total number of search queries received for the aggregated event. | `16023` | `Documents Searched by Multi-Search POST` |
177+
| indexes.avg_distinct_index_count | The average number of queried indexes received per call for the aggregated event. | `1.2` | `Documents Searched by Multi-Search POST`
178+
| indexes.total_distinct_index_count | The total number of distinct indexes queries for the aggregated event. | `6023` | `Documents Searched by Multi-Search POST`
179+
| indexes.total_single_index | The total number of calls where only one index where queried. | `2007` | `Documents Searched by Multi-Search POST`
172180
| `swap_operation_number` | The number of swap operation given in `POST /swap-indexes` API call | 2 | `Indexes Swapped` |
173181
| `matching_strategy.most_used_strategy` | Most used word matching strategy among all search requests in this batch | `last` | `Documents Searched POST`, `Documents Searched GET` |
174182
| `per_document_id` | `true` if `DELETE /indexes/:indexUid/documents/:documentUid` endpoint was used in this batch, otherwise `false` | false | `Documents Deleted` |
@@ -221,6 +229,7 @@ This property allows us to gather essential information to better understand on
221229
| infos.ssl_resumption | `true` if `--ssl-resumption`/`MEILI_SSL_RESUMPTION` is specified, otherwise `false` | false |
222230
| infos.ssl_tickets | `true` if `--ssl-tickets`/`MEILI_SSL_TICKETS` is specified, otherwise `false` | false |
223231
| infos.with_configuration_file | `true` if the instance is launched with a configuration file, otherwise `false` | `false` |
232+
| infos.experimental_enable_metrics | `true` if `--experimental-enable-metrics`/`MEILI_EXPERIMENTAL_ENABLE_METRICS` is specified at launch, otherwise `false` | `false` |
224233

225234
##### MeiliSearch Statistics `stats`
226235

@@ -251,6 +260,7 @@ This property allows us to gather essential information to better understand on
251260
| sort.with_geoPoint | Does the built-in sort rule _geoPoint rule has been used in the aggregated event? | `true` |
252261
| sort.avg_criteria_number | The average number of sort criteria among all the requests containing the `sort` parameter in the aggregated event. `"sort": []` equals to `0` while not sending `sort` does not influence the average. | `2` |
253262
| filter.with_geoRadius | Does the built-in filter rule _geoRadius has been used in the aggregated event? | `false` |
263+
| filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` |
254264
| filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` |
255265
| filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` |
256266
| q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
@@ -284,6 +294,7 @@ This property allows us to gather essential information to better understand on
284294
| sort.with_geoPoint | Does the built-in sort rule _geoPoint rule has been used in the aggregated event? | `true` |
285295
| sort.avg_criteria_number | The average number of sort criteria among all the requests containing the `sort` parameter in the aggregated event. `"sort": []` equals to `0` while not sending `sort` does not influence the average. | `2` |
286296
| filter.with_geoRadius | Does the built-in filter rule _geoRadius has been used in the aggregated event? | `false` |
297+
| filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` |
287298
| filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` |
288299
| filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` |
289300
| q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
@@ -303,6 +314,22 @@ This property allows us to gather essential information to better understand on
303314

304315
---
305316

317+
#### `Documents Searched by Multi-Search POST`
318+
319+
> The Documents Searched event is sent once an hour or when a batch reaches the maximum size of `500kb`. The event's properties are averaged over all requests on `POST` - `/multi-search`.
320+
321+
| Property name | Description | Example |
322+
|---------------|-------------|---------|
323+
| user_agent | Represents all the user-agents encountered on this endpoint in the aggregated event.| `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` |
324+
| requests.total_succeeded | The number of succeeded calls on the endpoint for the aggregated event. | `3456` |
325+
| requests.total_failed | The number of calls on the endpoint for the aggregated event. | `24` |
326+
| requests.total_received | The number of calls on the endpoint for the aggregated event. | `3480` |
327+
| searches.avg_search_count | The average number of search queries received per call for the aggregated event. | `4.2` |
328+
| searches.total_search_count | The total number of search queries received for the aggregated event. | `16023` |
329+
| indexes.avg_distinct_index_count | The average number of queried indexes received per call for the aggregated event. | `1.2`
330+
| indexes.total_distinct_index_count | The total number of distinct indexes queries for the aggregated event. | `6023`
331+
| indexes.total_single_index | The total number of calls where only one index where queried. | `2007`
332+
306333
## `Index Created`
307334

308335
| Property name | Description | Example |

0 commit comments

Comments
 (0)