Skip to content

exp/lighthorizon: Correctly paginate responses to respect query limits #4430

@sreuland

Description

@sreuland

What problem does your feature solve?

page limit parameter on requests should determine the response results sizee
contributes to #4317 development tasks, specifically to implement correct paging behavior for limit

What would you like to see?

A request to serve ?limit=N of a resource should actually serve N of that resource, rather than reading N ledgers and serving 0 <= M <= N instances matching the requested resource (start in ./archive/main.go).


Details

In ./archive/wrapper.go, we see two methods: GetTransactions() and GetOperations(). For both of these cases, the limit parameter is reached via the following (abridged) code:

txns = append(txns, common.Transaction{TransactionEnvelope: &tx.Envelope, /* etc. */})
if int64(len(txns)) == limit {
	return txns, nil
}

where tx comes from reader.Read(), an ingest.LedgerTransactionReader instance built from ledger. ledger comes from Wrapper.GetLedger(ledgerSequence) (where the sequence is derived from the cursor), and this corresponds to a lookup to a txmeta dump (see ./main.go).

Key point: We want to see indices utilized when finding ledgers to look up, rather than just using them as a starting point for the cursor (see actions/transaction.go: all the index store is used for is advancing cursor to the next active ledger). We should move index usage to within the GetTransactions() method and advance to the next active ledger, pulling transactions until the limit is fulfilled.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions