Skip to content

Track the cost of a subgraph #940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lutter opened this issue May 13, 2019 · 1 comment
Closed

Track the cost of a subgraph #940

lutter opened this issue May 13, 2019 · 1 comment
Labels
enhancement New feature or request wontfix This will not be worked on

Comments

@lutter
Copy link
Collaborator

lutter commented May 13, 2019

It would be useful to measure how much work a subgraph actually causes, e.g., so that we can make decisions on how to balance indexing of subgraphs across different graph-node instances.

To start, we should, for each block, look at the time that a subgraph spends running handlers plus the time it takes to store entities in the database as the cost for each block, and then compute a longer-term average using exponential decay, i.e. if the cost for one block is b_i, make the subgraph cost sum_{i=0}^n c^(n-i) * b_i for some c < 1; that measure would be easy to compute on a running basis, and the choice of c would determine over how many blocks we average the cost.

@leoyvens
Copy link
Collaborator

An important thing to define is if we want to measure the cost of syncing, which is done in block ranges, or the cost of updating which is done block by block.

It'd be great if we could time and register not only the total cost for a block (or range) but also the cost of each processing stage (after we rationalize what these stages are), this would help a lot in informing the investigation in #856 on how to speed up syncing.

@Jannis Jannis added enhancement New feature or request wontfix This will not be worked on labels Sep 16, 2020
@Jannis Jannis closed this as completed Sep 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

3 participants