feat(satp-hermes): gateway monitoring and dashboard #3876
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JJSantos22
changed the title
JJSantos22
changed the title
RafaelAPB
reviewed
Apr 25, 2025
| private async initializeMonitorService(): Promise<void> { | ||
| const fnTag = `${this.className}#initializeMonitorService()`; | ||
| try { | ||
| await this.monitorService.init(); |
There was a problem hiding this comment.
this functionality should be part of the monitoring service. In particular public methods that can be used throughout the code
RafaelAPB
force-pushed
the
satp-dev
branch
3 times, most recently
from
e28231a to
bba6481
Compare
|
As of this message (commit 745696c):
|
JJSantos22
force-pushed
the
Monitoring
branch
2 times, most recently
from
36db792 to
65167c3
Compare
RafaelAPB
force-pushed
the
satp-dev
branch
2 times, most recently
from
eb95768 to
1c1d902
Compare
Signed-off-by: Rafael Belchior <[email protected]> feat(bungee): add skeleton for bungee Signed-off-by: André Augusto <[email protected]> refactor(plugin-satp-hermes): update messages formats SATP core version 2.0.0 OpenAPI * created new messages in openAPI definition feat(bungee): fix bungee dependencies Signed-off-by: André Augusto <[email protected]> feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> test(SATP-Hermes): enable decorators needed for gateway coordinator checks Signed-off-by: Rafael Belchior <[email protected]> docs(SATP-Hermes): update package.json Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): remove kotlin sdk, add protobuffer Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): refactor naming Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure improvements Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): update yarn.lock and clean up Signed-off-by: André Augusto <[email protected]> refactor(test-tooling): fix types of streams: use NodeJS.ReadableStream 1. The container management library that we use in the test infrastructure (called dockerode) is expecting streams that are defined in the global namespace of the `@types/node` library, e.g. the standard library of NodeJS itself. 2. Previously we were using the "streams" package to provide type information to the streams that we were passing around to dockerode and it was working fine, but after some changes that seem unrelated this has broken the compilation process. 3. The mentioned changes are not yet on the main branch, but we expect them to be there soon and so this change is laying the groundwork for that by pre-emptively fixing the broken build's root cause which is that the test-tooling package does not declare it's typings related dependencies correctly: It implicitly uses the NodeJS standard library's types but so far had not declared them on the package level. 4. This change is therefore to rectify the issue of the `@types/node` dependency missing from the test-tooling package and also the refactoring of some of the test ledger classes which were relying on the `streams` builtin package instead of correctly using the NodeJS.ReadableStream global. 5. Earlier the reasoning for this was that we try to avoid pulling in types from the global scope because we try to avoid any sort of dependency on the global scope in general. Once we have proof though that this is causing issues with the build, then we must give up the principle for practical reasons (and only in the minimum viable scope, e.g. this does not change the fact that everywhere else in the codebase we should still do our best to avoid using the global scoped classes, types, functions, etc..). Thank you to @AndreAugusto11 and @RafaelAPB for pointing out this issue through the pull request of his that is currently being worked on at the time of this writing: RafaelAPB#72 Related to but does not address hyperledger-cacti#2811 Signed-off-by: Peter Somogyvari <[email protected]> refactor(SATP-Hermes): remove unused packages * updated openapi version * updated bungee plugin version to 2.0.0-alpha.2 Co-authored-by: Peter Somogyvari <[email protected]> Signed-off-by: André Augusto <[email protected]> Signed-off-by: Peter Somogyvari <[email protected]>
Co-authored-by: André Augusto <[email protected]> Co-authored-by: Peter Somogyvari <[email protected]> Co-authored-by: Rafael Belchior <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> test(satp-hermes): fix hanging tests Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): fix cspell and lint Signed-off-by: André Augusto <[email protected]>
Co-authored-by: Peter Somogyvari <[email protected]> Co-authored-by: André Augusto <[email protected]> Authored-by: Rafael Belchior <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
Authored by: Carlos Amaro <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): satp proto update (hyperledger-cacti#3147) * feat(satp-hermes): update protos Signed-off-by: Carlos Amaro <[email protected]> * feat(satp-hermes): protos Signed-off-by: Carlos Amaro <[email protected]> * feat(satp-hermes): protos Signed-off-by: Carlos Amaro <[email protected]> --------- Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): update protos Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): core stage services message names update Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): gRPC server start implementation and gateway_refactor in handlers Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): satp proto update also updates yarn lock Signed-off-by: Rafael Belchior <[email protected]> fix(satp-hermes): update yarn, comment method fix(satp-hermes): update yarn ci(satp-hermes): fix bungee and satp ci Signed-off-by: Rafael Belchior <[email protected]> chore(satp-hermes): delete Kotlin SDK, add Go SDK Signed-off-by: Rafael Belchior <[email protected]> build(satp-hermes): add gateway client Go SDK Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): re-add message proto Signed-off-by: Rafael Belchior <[email protected]> fixup(satp-hermes): remove test protos and endpoints Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): stage1-service and handler refactored Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes):stage2-service and handler refactored server-side Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): service handlers server Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): client stage handlers refactor Signed-off-by: Carlos Amaro <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Carlos Amaro <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]> Co-authored-by: Carlos Amaro <[email protected]> fix: refactor services and handlers Signed-off-by: Rafael Belchior <[email protected]> fix: eslint all Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): fabric and besu bridges, wrapper contracts with ontology (hyperledger-cacti#3382) feat(satp-hermes): added forge to compile solidity contracts Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): added generated code (hyperledger-cacti#3443) Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): run sequencial instead of paralel (hyperledger-cacti#3445) Signed-off-by: Carlos Amaro <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Iulia Mihaiu <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): add methods to fabric connector Signed-off-by: Iulia Mihaiu <[email protected]> fix(prometheus): metrics.ts leaks to global registry hyperledger-cacti#1202 1. Specified a `register` property of the gauges as an empty array so that it does not pollute the global namespace. This is how this is supposed to be done as per the docs of prom-client. 2. Once the change from 1) took place, the issue became that the metrics gathering code was still trying to hit up the global scope for the metrics, e.g. calling the get metrics methods directly on the promClient object instead of the registry that we create for each prmoetheus exporter object separately. So a little additional refactor ensued to fix this as well by making sure that we grab a reference of the registry object at construction time and then re-use that wherever needed instead of going through the global promClient object. 3. Added missing .startMetricsCollection calls in the plugin constructors to ensure that the prometheus exporter object gets initialized properly (this is where the registry gets created as well so without this there are crashes happening when one tries to access the metrics through the registry) Why though? The problem was that the metrics.ts file that we have for all the plugin's metrics constructs a new Metric (Gauge) object at import time which then defaults to registering the metric in the global registry of prom-client by default. The latter was causing crashes when separate versions of the same metrics.ts file are imported in a scenario were the API server imports plugins from a different directory (this issue is coming from the branch where I'm working on plugin sandboxing via the live-plugin-manager). Fixes hyperledger-cacti#1202 Signed-off-by: Peter Somogyvari <[email protected]> feat(cc-tx-viz): added test for multiple prometheus metrics on fabric Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): add metric model Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): added croschainEventLog models Signed-off-by: Iulia Mihaiu <[email protected]> feat(fabric): added endpoint for list of transaction receipts Signed-off-by: Iulia Mihaiu <[email protected]> feat(test-tooling): add RabbitMQ test server feat(cc-tx-viz): add RabbitMQ support; update model feat(cc-tx-viz): add preliminary support to Fabric and Besu build(deps): upgrade axios to latest to fix CVE Details CVE-2021-3749 high severity Vulnerable versions: <= 0.21.1 Patched version: 0.21.2 axios is vulnerable to Inefficient Regular Expression Complexity Signed-off-by: Peter Somogyvari <[email protected]> feat(cc-tx-viz): add methods to fabric connector Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): add metric model Signed-off-by: Iulia Mihaiu <[email protected]> xfix(cc-tx-viz): error implementation of IsVisualizable feat(cc-tx-viz): add methods to fabric and besu; update models Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): update cross chain events log for fabric transactions Signed-off-by: Iulia Mihaiu <[email protected]> feat(cctxviz): fix fabric prometheus feat(cctxviz): fix besu prometheus feat(cctxviz): fix fabric test feat(cctxviz): update tx receipt feat(cctxviz): add test script Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add test receipts feat(cctxviz): refactor CrossChainEventLog Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): refactor tests feat(cc-tx-viz): update model, add tests chore(cc-tx-viz): update dependencies Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): persist logs Signed-off-by: Rafael Belchior <[email protected]> test(cc-tx-viz): update test Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): update strinfify to csv Signed-off-by: Rafael Belchior <[email protected]> chore(cc-tx-viz): add csv folder Signed-off-by: Rafael Belchior <[email protected]> chore(cc-tx-viz): move csv folder Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): add test feat(cctxviz): add csv file, process mining script Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): proof of concept v1 Signed-off-by: Rafael Belchior <[email protected]> chore(cctxviz): update csv Signed-off-by: Rafael Belchior <[email protected]> docs(cctxviz): update readme Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update tests Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add carbon footprint Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update tests Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update model Signed-off-by: Rafael Belchior <[email protected]> feat(fabric-connector): update default queue Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): fabric test Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add parameters to fabric calls Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): support besu Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update metrics Signed-off-by: Rafael Belchior <[email protected]> test(cctxvi): update fabric and besu tests Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): besu and fabric Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): support call transactions on fabric Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add aggregation basis Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): aggregate cctx Signed-off-by: Rafael Belchior <[email protected]> chore: update yarn and remove package.lock Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): containerize draft Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add revenue Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update model playbook Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add dummy use case playbook Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update persist test and dummy uc Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add timer Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): workload test Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add more timers Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add timer to map on connectors Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update testing framework Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): vis plot wip Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): evaluate baseline Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update plots Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add test files Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update testing framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): time-efficient polling receipts Signed-off-by: Rafael Belchior <[email protected]> chore(cctxviz): add test chaincode Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update test Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): use case v1 Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update test framework Signed-off-by: Rafael Belchior <[email protected]> build(cc-tx-visualization): added missing fabric-contract-api This will fix one compilation error. Signed-off-by: Peter Somogyvari <[email protected]> chore(deps): updated yarn.lock after rebase onto upstream/main Signed-off-by: Peter Somogyvari <[email protected]> build(cc-tx-visualization): deleted redundant test fixtures Also added the test-tooling package as a dev dependency which should fix a few of the compiler errors we were getting on account of missing dependencies. Signed-off-by: Peter Somogyvari <[email protected]> fix: resolved build errors for cctxviz branch - test-tooling package changed that needs to be reverted before commmit to main branch - added import statement for test data generationt xfix(cc-tx-viz): error implementation of IsVisualizable chore: update yarn and remove package.lock Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): testing framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): test framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add graphs Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add test Signed-off-by: Rafael Belchior <[email protected]> fix(cctxviz): fix bug Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add process conformance Signed-off-by: Rafael Belchior <[email protected]>
The plugin now utilizes RxJS instead of RabbitMQ in transaction monitoring. ReplaySubjects store and emit observed transactional data to subscribers. fix(cc-tx-visualization): rebase latest version Signed-off-by: Rafael Belchior <[email protected]> squash! - migrate a test case to Fabric v2.5.6 LTS AIO This is just a snippet of a change from the pair programming call with Bruno, it can be safely squashed, no worries. Signed-off-by: Peter Somogyvari <[email protected]> Signed-off-by: Bruno Mateus <[email protected]> feat(cc-tx-visualization): ccmodel serialization with PM4PY The plugin now creates the ccmodel for conformance checking with PM4PY. Signed-off-by: Bruno Mateus <[email protected]>
The plugin is now able to: Create a Cross-Chain Model based on monitored transactions Compare new transactions against the model and detect non-conformance Signed-off-by: Bruno Mateus <[email protected]>
JJSantos22
force-pushed
the
Monitoring
branch
2 times, most recently
from
3265a91 to
5e9f2ca
Compare
|
As of this message (commit 5e9f2ca):
|
JJSantos22
force-pushed
the
Monitoring
branch
3 times, most recently
from
0826e54 to
54c31d0
Compare
RafaelAPB
force-pushed
the
satp-stg
branch
2 times, most recently
from
a31aaf2 to
88b135f
Compare
Signed-off-by: Andre Augusto <[email protected]> fix(satp-hermes): update networkid Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]>
Authored-by: Andre Augusto <[email protected]> Squash with previous commit: ci(satp-hermes): ci fix (hyperledger-cacti#3896) Signed-off-by: Carlos Amaro <[email protected]> Co-authored-by: Carlos Amaro <[email protected]> Co-authored-by: Rodolfo Carapau <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
ci(satp-hermes): cdbc Signed-off-by: Rodolfo Carapau <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
Add foundry tests for solidity (hyperledger-cacti#3929) Signed-off-by: Andre Augusto <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
Signed-off-by: Carlos Amaro <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>
JJSantos22
force-pushed
the
Monitoring
branch
3 times, most recently
from
1d59e89 to
3e52e64
Compare
JJSantos22
force-pushed
the
Monitoring
branch
2 times, most recently
from
1ea77ba to
59b409a
Compare
Signed-off-by: Rafael Belchior <[email protected]>
…-cacti#3951) fix(satp-hermes): fixed session utils and session hashes Signed-off-by: Rodolfo Carapau <[email protected]>
Signed-off-by: Jorge Santos <[email protected]>
RafaelAPB
force-pushed
the
satp-stg
branch
2 times, most recently
from
59c561e to
65e385e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix(satp-hermes): corrected docker-compose file to create correctly the grafana dashboard;
feat(satp-hermes): added healthcheck-endpoint metrics (Not working);