Skip to content

add workflow to simulate node restarts using pumba #2119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ffakenz
Copy link
Contributor

@ffakenz ffakenz commented Jul 8, 2025

Randomly restarts hydra-node containers during benchmarks to observe system resilience and network recovery behavior.


  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

@ffakenz ffakenz requested a review from a team July 8, 2025 21:56
@ffakenz ffakenz self-assigned this Jul 8, 2025
@github-project-automation github-project-automation bot moved this to Triage 🏥 in ☕ Hydra Team Work Jul 8, 2025
Copy link

github-actions bot commented Jul 8, 2025

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2025-07-15 13:39:53.835188272 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045 2652
νCommit 61458bc2f297fff3cc5df6ac7ab57cefd87763b0b7bd722146a1035c 685
νHead be6ebc744208c660bf0fdc1cfbb5157477cd305de5b1777e575cbb4c 14665
μHead 1f47a42d1d6edc32ccd834acb19d5db3b2a5232f0bd7eaa8908dc519* 5284
νDeposit ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c 1102
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 5836 10.19 3.22 0.51
2 6037 12.46 3.94 0.55
3 6239 14.72 4.66 0.58
5 6640 18.43 5.81 0.63
10 7648 29.12 9.18 0.79
43 14285 98.73 30.85 1.80

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 559 2.44 1.16 0.20
2 739 3.38 1.73 0.22
3 923 4.36 2.33 0.24
5 1280 6.41 3.60 0.28
10 2173 12.13 7.25 0.40
54 10060 98.61 68.52 1.88

CollectCom transaction costs

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 525 24.43 7.12 0.42
2 114 636 34.31 9.88 0.53
3 170 747 43.59 12.49 0.63
4 226 858 51.97 14.89 0.72
5 283 974 61.14 17.48 0.81
6 339 1081 65.99 19.00 0.87
7 397 1192 78.65 22.46 1.00
8 449 1303 83.35 24.08 1.05
9 504 1414 93.68 26.90 1.16

Cost of Increment Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 1748 23.20 7.66 0.47
2 1938 25.24 9.07 0.50
3 2104 27.88 10.62 0.54
5 2545 32.61 13.63 0.62
10 3076 38.33 18.61 0.73
40 7375 92.88 56.21 1.63

Cost of Decrement Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 633 22.80 7.37 0.42
2 758 24.25 8.44 0.44
3 977 26.93 9.85 0.48
5 1230 29.09 11.77 0.52
10 2184 42.84 18.97 0.73
41 6285 91.20 53.07 1.55

Close transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 708 27.33 8.81 0.47
2 740 30.15 10.16 0.50
3 1019 31.36 11.49 0.53
5 1269 34.75 13.87 0.59
10 2079 48.27 21.69 0.79
37 5935 96.18 55.11 1.59

Contest transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 628 33.02 10.19 0.52
2 808 35.65 11.76 0.56
3 995 38.45 13.37 0.60
5 1295 42.43 15.94 0.66
10 2013 53.69 22.88 0.84
29 4997 98.47 49.98 1.53

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 5823 27.09 9.09 0.69
2 5902 34.91 11.69 0.78
3 6162 45.69 15.41 0.90
4 6172 50.34 16.86 0.95
5 6443 63.88 21.53 1.11
6 6619 71.36 24.02 1.19
7 6678 78.57 26.39 1.27
8 6783 88.65 29.75 1.38

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTXO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
10 0 0 5834 19.64 6.56 0.61
10 5 285 6004 28.92 10.28 0.72
10 10 570 6174 38.83 14.22 0.84
10 20 1138 6512 59.57 22.38 1.08
10 30 1705 6851 80.53 30.62 1.32
10 39 2220 7159 98.99 37.90 1.54

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2025-07-15 13:44:24.62940793 UTC

Baseline Scenario

Number of nodes 1
Number of txs 300
Avg. Confirmation Time (ms) 4.312965576
P99 10.907685739999993ms
P95 5.101238800000001ms
P50 4.055712ms
Number of Invalid txs 0

Memory data

Time Used Free
2025-07-15 13:43:09.024184223 UTC 0m 0;
2025-07-15 13:43:14.024089504 UTC 13 0;
2025-07-15 13:43:19.023958568 UTC 5 0;
2025-07-15 13:43:24.02396027 UTC 1 0;
2025-07-15 13:43:29.023923484 UTC 1 0;
2025-07-15 13:43:34.024046736 UTC 1 0;

Three local nodes

Number of nodes 3
Number of txs 900
Avg. Confirmation Time (ms) 27.458376682
P99 42.65176027ms
P95 35.95386865ms
P50 26.390521ms
Number of Invalid txs 0

Memory data

Time Used Free
2025-07-15 13:43:47.498015825 UTC 0m 0;
2025-07-15 13:43:52.498634236 UTC 11 0;
2025-07-15 13:43:57.498205229 UTC 5 0;
2025-07-15 13:44:02.498138877 UTC 22 0;
2025-07-15 13:44:07.498078692 UTC 14 0;
2025-07-15 13:44:12.498090064 UTC 2 0;
2025-07-15 13:44:17.49819017 UTC 2 0;
2025-07-15 13:44:22.498028287 UTC 2 0;

Copy link

github-actions bot commented Jul 8, 2025

Transaction cost differences

No cost or size differences found

@ffakenz ffakenz marked this pull request as ready for review July 8, 2025 22:29
@ffakenz ffakenz moved this from Triage 🏥 to In review 👀 in ☕ Hydra Team Work Jul 8, 2025
@ffakenz ffakenz force-pushed the network-test-restart branch 3 times, most recently from 3c42df9 to 8a99d0d Compare July 15, 2025 09:26
ffakenz added 2 commits July 15, 2025 15:38
Randomly restarts hydra-node containers during benchmarks
to observe system resilience and network recovery behavior.
@ffakenz ffakenz force-pushed the network-test-restart branch from 8a99d0d to 31ab8f4 Compare July 15, 2025 13:38
@ffakenz ffakenz changed the base branch from master to persistent-queue July 15, 2025 13:40
@ffakenz ffakenz changed the base branch from persistent-queue to master July 15, 2025 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In review 👀
Development

Successfully merging this pull request may close these issues.

Reproduce issue where hydra-node crashes before processing item off the queue
1 participant