Description
Limits
AFAIK, there's no notion of result limits in grafana, the pagination only relies on timestamps and the amount of results expected for a given timerange is unbounded. There's an edge case to this mode where a lot of logs have the same timestamp. If the datasource doesn't limit the results, the plugin may explode (could be an attack vector) . If the datasource puts a limit, logs may be lost between pages or the pagination may fail to compute bounds for the next page.
IMO, Grafana should implement a more robust pagination scheme to solve this case (we should probably open an issue on that), but we could work around it by implementing unbounded DataQueries as a sequence of bounded ones (the result is an observable). Not ideal, but it would avoid losing logs and allow throttling strategies to mitigate the risk of DoS.
Sorting
Grafana allows the user to sort logs results ascending or descending and adapts querying actions accordingly. This SHOULD be enough to cover both use cases of chronological and backwards exploration.
However, the limit on results interferes as we don't specify if the results should be fetched from the beginning or the end of the specified range.
Pagination
Up to Grafana 10.3.x, the pagination component tends to break and fail to recognize which page it's in. Current main branch fixes that bug and implements infinite scrolling as an experimental feature.
Msearch API
Each query in a multi-search request defines its own limits, pagination must take multi-requests into account.
See also:
- Known edge case when a lot of logs have the same timestamp
Logs pagination: Older logs button stops to work if start and end timestamp are the same grafana/grafana#64041 - Infinite scrolling feature (feature toggle :
logsInfiniteScrollingin 10.4)
Logs: Initial setup to add infinite scrolling to Explore grafana/grafana#76348 - Pagination bug fixed in 10.4
Logs navigation: derive current page index grafana/grafana#80167