Skip to content

Bridges V2 refactoring backport and pallet_bridge_messages simplifications#4935

Merged
bkontur merged 64 commits into
masterfrom
bko-bridges-v2-backport-refactoring
Jul 12, 2024
Merged

Bridges V2 refactoring backport and pallet_bridge_messages simplifications#4935
bkontur merged 64 commits into
masterfrom
bko-bridges-v2-backport-refactoring

Conversation

@bkontur
Copy link
Copy Markdown
Contributor

@bkontur bkontur commented Jul 3, 2024

Summary

This PR contains migrated code from the Bridges V2 branch from the old parity-bridges-common repo.
Even though the PR looks large, it does not (or should not) contain any significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were implemented on top of these changes.

TODO

Relates to: paritytech/parity-bridges-common#2976
Relates to: paritytech/parity-bridges-common#2451

serban300 and others added 29 commits June 26, 2024 21:08
* Define Chain::STATE_VERSION

* Add vec_db module

* Use VecDb instead of StorageProof for message delivery

* Make sure that the TrustedVecDb is sorted

* Address review comments

* Run benchmarks on parent commit

* Run benchmarks with new code

* Fix test

* Fix code review comments
* moved FromBridgedChainMessagesDeliveryProof to bp-messages

* spelling

* fix benchmarks compilation
* moved FromBridgedChainMessagesProof to bp-messages

* fmt
* moved message files to separate folder

* ...and fix paths in code

* fmt

rebase nit
* Adjust messages pallet benchmarks

* Address comment
* moved files around

* and fix compilation

Move Chain::ID from relay-level Chain to primitives-level Chain (#2181)

* move Chain::ID from relay-level Chain to primitives-level Chain

* removed chain IDs from bp-runtime

* add missing file header

Adjust weights (#2185)

fix clippy (#2186)

Remove source header chain (#2183)

* moved messages proof verification to messages pallet

* removed SourceHeaderChain

* cleanup

* benchmarks compilation

* fix benchmark tests compilation

* clippy

* fmt

Use compact proofs for messages delivery confirmation (#2187)

* Use compact proofs for messages delivery confirmation

* Fix benchmarks

Move messages delivery proof verification to pallet (#2189)

* rename messages_proof.rs to proofs.rs

* moved delivery proof verification to the messages pallet

* removed TargetHeaderChain

* cleaning up

* fixed benchmarks compilation

* Update modules/messages/README.md

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* uncommented test and removed printlns

* vec![].into_iter().collect() -> vec![].into()

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config (#2190)

* removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config

* fixed doc

* fixed benchmarking code

* more fixes

remove MaximalOutboundPayloadSize and use BridgedChain::maximal_incomging_message_size instead (#2191)

remove MessageBridge trait (#2192)
* Use compact proofs for parachains

* Remove StorageProofChecker

* Cleanup

Reuse messages proof generation from messages pallet (#2195)

* reuse messages proof generation from messages pallet

* incorrect merge

fixed remaining TODOs after messages pallet Config refactoring (#2196)

Return UntrustedVecDb from prove_storage() (#2197)

Storage proofs related renamings (#2198)

* Storage proofs related renamings

* Leftovers

* StorageSize -> StorageProofSize
* add test for maximal message dispatch weight

* fixed weight computations - use kb for both size-related benchmarks

* fix typo in weight formula

* decreased maximal message size to 64Kb
* Box messages proof argument in receive_messages_proof call

* fix benchmark tests
* StorageProofSize -> StorageSize

* Rename benchmarks

* StorageSize -> UnverifiedStorageProofParams

* Fix clippy

* Fix priority boost
* update async-trait to 0.1.69

* more clippy fixes
* some code grooming: enable warn(missing_docs) for all piblic crates + added missing documentation + removed obsolete clippy/deny workarounds

* removed strange allow + added comment related to other allow

* removed incorrect_clone_impl_on_copy_type which is unknown to CI clippy
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
@bkontur bkontur added the T15-bridges This PR/Issue is related to bridges. label Jul 3, 2024
…dle (#4944)

Original PR with more context:
paritytech/parity-bridges-common#2211
Relates to:
paritytech/parity-bridges-common#2210

## TODO

- [x] fresh weighs for `pallet_bridge_messages`
- [x] add `try_state` for `pallet_bridge_messages` which checks for
unpruned messages - relates to the
[comment](paritytech/parity-bridges-common#2211 (comment))
- [x] ~prepare migration, that prunes leftovers, which would be pruned
eventually from `on_idle` the
[comment](paritytech/parity-bridges-common#2211 (comment)
can be done also by `set_storage` / `kill_storage` or with
`OnRuntimeUpgrade` implementatino when `do_try_state_for_outbound_lanes`
detects problem.

## Open question

- [ ] Do we really need `oldest_unpruned_nonce` afterwards?
- after the runtime upgrade and when `do_try_state_for_outbound_lanes`
pass, we won't need any migrations here
    - we won't even need `do_try_state_for_outbound_lanes`

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
Copy link
Copy Markdown
Contributor

@acatangiu acatangiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good!

Should do follow-up investigation on grandpa submit proof weight for bridge-hub-rococo.

Comment thread bridges/modules/messages/src/weights.rs
Comment thread bridges/modules/parachains/src/weights.rs
@bkontur
Copy link
Copy Markdown
Contributor Author

bkontur commented Jul 11, 2024

bot clean
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa

@command-bot
Copy link
Copy Markdown

command-bot Bot commented Jul 11, 2024

@bkontur https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 2-69bc0eb3-a7d2-4a77-9a68-35bf29cea43c to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link
Copy Markdown

command-bot Bot commented Jul 11, 2024

@bkontur https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 3-a676174b-7a34-4bc3-871b-24cfd744bc6b to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot added 2 commits July 11, 2024 12:25
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link
Copy Markdown

command-bot Bot commented Jul 11, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444/artifacts/download.

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link
Copy Markdown

command-bot Bot commented Jul 11, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445/artifacts/download.

@bkontur bkontur enabled auto-merge July 11, 2024 14:16
@bkontur bkontur added this pull request to the merge queue Jul 11, 2024
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 11, 2024
@bkontur bkontur enabled auto-merge July 12, 2024 07:25
@bkontur bkontur added this pull request to the merge queue Jul 12, 2024
Merged via the queue into master with commit 1f8e448 Jul 12, 2024
@bkontur bkontur deleted the bko-bridges-v2-backport-refactoring branch July 12, 2024 08:48
bkontur added a commit to paritytech/parity-bridges-common that referenced this pull request Jul 12, 2024
…k#4935 (#3022)

* Companion for paritytech/polkadot-sdk#4935

* Fix imports for `FromBridgedChainMessagesProof` / `FromBridgedChainMessagesDeliveryProof`

* Align with `polkadot-sdk`

* Align with actual `polkadot-sdk`

* Align with actual `polkadot-sdk`

* fmt

* Undienerize back to master

* Update tools/runtime-codegen
TomaszWaszczyk pushed a commit to TomaszWaszczyk/polkadot-sdk that referenced this pull request Jul 13, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
ordian added a commit that referenced this pull request Jul 15, 2024
* master: (120 commits)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  fixed cmd bot commenting not working (#5000)
  Explain usage of `<T: Config>` in FRAME storage + Update parachain pallet template  (#4941)
  Expose metadata-hash feature from polkadot crate (#4886)
  Add `MAX_INSTRUCTIONS_TO_DECODE` to XCMv2 (#4978)
  add notices to the implementer's guide docs that changed for elastic scaling (#4983)
  `polkadot-parachain` simplifications and deduplications (#4916)
  Update Templates README docs (#4980)
  allow clear_origin in safe xcm builder (#4777)
  litep2p/peerstore: Fix bump last updated time (#4971)
  Make `tracing::log` work in the runtime (#4863)
  sp-core: Improve docs generated by `generate_feature_enabled_macro` (#4968)
  [Backport] Version bumps  and  prdocs reordering from 1.14.0 (#4955)
  Assets: can_decrease/increase for destroying asset is not successful (#3286)
  ...
ordian added a commit that referenced this pull request Jul 18, 2024
* master: (125 commits)
  add elastic scaling MVP guide (#4663)
  Send PeerViewChange with high priority (#4755)
  [ci] Update forklift in CI image (#5032)
  Adjust base value for statement-distribution regression tests (#5028)
  [pallet_contracts] Add support for transient storage in contracts host functions (#4566)
  [1 / 5] Optimize logic for gossiping assignments (#4848)
  Remove `pallet-getter` usage from pallet-session (#4972)
  command-action: added scoped permissions to the github tokens (#5016)
  net/litep2p: Propagate ValuePut events to the network backend (#5018)
  rpc: add back rpc logger (#4952)
  Updated substrate-relay version for tests (#5017)
  Remove most all usage of `sp-std` (#5010)
  Use sp_runtime::traits::BadOrigin (#5011)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  ...
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
ordian added a commit that referenced this pull request Aug 6, 2024
* master: (130 commits)
  add elastic scaling MVP guide (#4663)
  Send PeerViewChange with high priority (#4755)
  [ci] Update forklift in CI image (#5032)
  Adjust base value for statement-distribution regression tests (#5028)
  [pallet_contracts] Add support for transient storage in contracts host functions (#4566)
  [1 / 5] Optimize logic for gossiping assignments (#4848)
  Remove `pallet-getter` usage from pallet-session (#4972)
  command-action: added scoped permissions to the github tokens (#5016)
  net/litep2p: Propagate ValuePut events to the network backend (#5018)
  rpc: add back rpc logger (#4952)
  Updated substrate-relay version for tests (#5017)
  Remove most all usage of `sp-std` (#5010)
  Use sp_runtime::traits::BadOrigin (#5011)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  ...
pandres95 added a commit to pandres95/runtimes that referenced this pull request Oct 19, 2024
sfffaaa pushed a commit to peaqnetwork/polkadot-sdk that referenced this pull request Dec 27, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T15-bridges This PR/Issue is related to bridges.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants