Custom Analytics endpoint for telemetry data

[RickyWanga]

It would be nice to have a parameter in the meilisearch config.toml for setting up a custom url endpoint, where the telemetry data can be sent to.
It could be use for collecting statistics regarding queries, filters, facets, etc... such as:
-Total search requests
-Total number of results returned for each query
-Most searched queries
-Most used filters/facets
-Average result count by query
-Most used syntax
-Highest number of terms given for the query parameter
-Average number of facets
-The average number of search queries received per call for the aggregated event
-The average number of queried indexes received per call for the aggregated event
-Total number of distinct facets queried for the aggregated event
etc....

Basically more detailed analytics regarding queries and indexes.

[gmourier]

Hey @RickyWanga

Thanks for reaching us.

Most of the data points described here are separated into three aspects/features.

• Search analytics/insights are available via our cloud product.
  • The engine doesn't compute or send anything specific. All is done on the cloud infrastructure.
• You can use the experimental /metrics endpoint to obtain some of the information listed here to monitor an instance.
• Finally, some of the information described here is also sent internally via segment so we can measure API feature usage and feed data into our product decisions. You can find a described list here.
  • We could add a custom endpoint for these in the future. Could it help you to achieve something in particular to have these?

Today, it would take too much time for our team to work and think of a centralized way to export them; in addition to that, we have other priorities to tackle for now.

I'm leaving this discussion open so that other users can testify to their needs, and we can re-assess that properly in the future.

[RickyWanga]

Yes, I was mainly interested in segment analytics, it would really be useful for us to be able to have analytics mainly regarding the search queries and facets, so we can monitor which queries yield more results and what are the most used facets.
One use case might be an e-commerce platform gathering insights regarding most sought after products and trends.

The addition of said endpoint would really make meilisearch easier to ship to production.