-
Notifications
You must be signed in to change notification settings - Fork 2.7k
Conversation
@pepyakin could you rebase this? |
5bdb54d
to
dddaefb
Compare
dddaefb
to
e89b8aa
Compare
it's pretty huge - would be good to get a few comments in here to describe what it's meant to do how it's meant to work and which bits do what. |
e89b8aa
to
6e01451
Compare
|
||
#[test] | ||
fn imports() { | ||
// nothing can be imported from non-"env" module for now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a new test that ensures we correctly check signatures.
@@ -404,4 +412,71 @@ mod tests { | |||
).unwrap(); | |||
assert_eq!(return_val, ReturnValue::Value(TypedValue::I32(0x1337))); | |||
} | |||
|
|||
#[test] | |||
fn signatures_dont_matter() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the test that actually ensures that libsandbox actually doesn't care of signatures.
//! This module provides a means for executing contracts | ||
//! represented in wasm. | ||
|
||
use exec::{CallReceipt, CreateReceipt}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this code mostly preserved from src/vm.rs
// along with Substrate. If not, see <http://www.gnu.org/licenses/>. | ||
|
||
//! Module that takes care of loading, checking and preprocessing of a | ||
//! wasm module before execution. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this code mostly preserved from src/vm.rs
the difference in that instead of scanning imports for imports of memory instances we now also process all functions, checking their signatures.
@gavofyork Yeah, sure! IntroductionWe have a library
This leads to the fact that it's up to the supervisor to check if the given module is compatible with the environment. That is, the owner of the sandbox should check that all imported functions not only exists but also have proper signatures. That was a long-standing design problem that I delayed to implement as much as possible, since I don't find this design particulary user-friendly. With all that, here is some problems with the former code:
The Pull RequestWe can't really get rid of The mechanism is very similar to the one we have in substrate-executor: we have a macro that allows us to declare some set of external functions with their signatures and bodies. In the bodies of these functions you can only access arguments declared in the signature and return the value that specified by the signature (example). Under the hood However, in the case with the substrate-executor import signatures are checked by the wasmi. And with the libsandbox we have to do it by ourselves. We already have been doing some prepairatory things, so this checks are fit naturally in preparing code. As I've noted in the PR description, I split the src/vm.rs into several modules: |
36c81b2
to
d24ebdd
Compare
d24ebdd
to
0a8858e
Compare
#[test] | ||
fn imports() { | ||
// nothing can be imported from non-"env" module for now. | ||
let r = parse_and_prepare_wat(r#"(module (import "vne" "memory" (memory 1 1)))"#); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"vne"
looks too much like an actual typo - maybe rename to "another_module"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed!
Since "vne" looks like an actual typo
…and-rlpcodec * master: Contract signatures checking (#478) extrinsic-pool: use retain() (#613) rename Polkadot to Substrate in the license header via following four commands (#614) typo fixes (#608) RPC: Block number to block hash (#584) Minor fixes for nightly 2018-08-18 (#600) Time-based transaction banning (#594) (#602) cargo --force to allow CI to build. (#599) Fix logging (#587) Fix runtime version cache (#586)
update substrate to ab823cf
* Add Proxy module * Cargo fmt * Bump spec version
…ntial-writes Batch writing piece indexes to file db for sequential piece writing
* metadata: Implement MetadataHashable for deterministic hashing Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Hash `scale_info::Field` Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Hash `scale_info::Variant` Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Hash `scale_info::TypeDef` Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Hash pallet metadata Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Avoid data representation collision via unique identifiers Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Finalize hashing on recursive types Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Cache recursive calls Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Move `MetadataHashable` to codegen to avoid cyclic dependency Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Add pallet unique hash Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Wrap metadata as owned Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Use MetadataHashable wrapper for clients Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Generate runtime pallet uid from metadata Signed-off-by: Alexandru Vasile <[email protected]> * Validate metadata compatibility at the pallet level Signed-off-by: Alexandru Vasile <[email protected]> * Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * Modify examples and tests for the new API Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Implement metadata uid Signed-off-by: Alexandru Vasile <[email protected]> * Update polkadot with TryFrom implementation Signed-off-by: Alexandru Vasile <[email protected]> * client: Change `to_runtime_api` to reflect TryFrom changes Signed-off-by: Alexandru Vasile <[email protected]> * client: Skip full metadata validation option Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Add option to skip pallet validation for TransactionApi Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Add option to skip pallet validation for StorageApi Signed-off-by: Alexandru Vasile <[email protected]> * Update polkadot.rs with ability to skip pallet validation Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Change `MetadataHashable` to per function implementation Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Use metadata hashes functions Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Use metadata hashes functions Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Make `get_type_uid` private Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Rename metadata functions `*_uid` to `*_hash` Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Update `get_field_hash` to use `codec::Encode` Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Move metadata check from client to subxt::Metadata Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Rename metadata check functions to follow `*_hash` naming Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Update polkadot.rs to reflect naming changes Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Use `encode_to` for metadata generation Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Update polkadot.rs to reflect `encode_to` changes Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Specific name for visited set Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Provide cache to hashing functions Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Compute metadata hash by sorted pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Get extrinsic hash Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Extend metadata hash with extrinsic and metadata type Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add cache as metadata parameter Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Update metadata hash to use cache Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Implement Default trait for MetadataHasherCache Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add cache for pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Move functionality to metadata crate Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Use subxt-metadata crate Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Remove metdata hashing functionality Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add documentation Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix vector capacity to include extrinisc and type hash Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add empty CLI Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Fetch metadata from substrate nodes Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Log metadata hashes of provided nodes Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Group compatible nodes by metadata Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Simplify hash map insertion Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Move full metadata check to function Signed-off-by: Alexandru Vasile <[email protected]> * metadata-cli: Group metadata validation at the pallet level Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Persist metadata cache Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Move compatibility cli from subxt-metadata to subxt-cli Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Remove cli from subxt-metadata Signed-off-by: Alexandru Vasile <[email protected]> * cli: Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix compatible metadata when pallets are registered in different order Signed-off-by: Alexandru Vasile <[email protected]> * tests: Handle result of pallet hashing Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Remove type cache for deterministic hashing Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add test assets from `substrate-node-template` tag `polkadot-v0.9.17` Signed-off-by: Alexandru Vasile <[email protected]> * metadata-tests: Check cache hashing for Balances pallet Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix `get_type_hash` clippy issue Signed-off-by: Alexandru Vasile <[email protected]> * metadata-tests: Compare one time cache with persistent cache Signed-off-by: Alexandru Vasile <[email protected]> * metadata-test: Check metadata hash populates cache for pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata-tests: Simplify `cache_deterministic_hash` test Signed-off-by: Alexandru Vasile <[email protected]> * metadata-tests: Check deterministic metadata for different order pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Implement TransactionApiUnchecked for skipping pallet validation Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Implement StorageApiUnchecked for skipping pallet validation Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Remove skip_pallet_validation boolean Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Implement ClientUnchecked for skipping metadata validation Signed-off-by: Alexandru Vasile <[email protected]> * Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * Update examples of rpc_call to skip metadata Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Remove heck dependency Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Add pallet name as an identifier for pallet hashing Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Implement MetadataHashDetails Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Adjust testing to `MetadataHashDetails` interface Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Remove extra `pallet_name` Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Fix clippy issue Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Change StorageApi to support `_unchecked` methods Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Change TransactionApi to support `_unchecked` methods Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Switch back from `TryFrom` to `From` for `subxt::Client` Signed-off-by: Alexandru Vasile <[email protected]> * codegen, subxt: Remove `ClientUnchecked` Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Expose `validate_metadata` as validation of compatibility method Signed-off-by: Alexandru Vasile <[email protected]> * examples: Update to the new interface Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Update test integration to latest interface Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Check different pallet index order Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Check recursive type hashing Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Check recursive types registered in different order Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Fix recursive types warning Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Remove test assets Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Extend tests to verify cached pallet values Signed-off-by: Alexandru Vasile <[email protected]> * examples: Add metadata compatiblity example Signed-off-by: Alexandru Vasile <[email protected]> * examples: Revert balance_transfer to initial form Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Add ConstantsApi metadata check Signed-off-by: Alexandru Vasile <[email protected]> * tests: Modify tests to accomodate ConstantsApi changes Signed-off-by: Alexandru Vasile <[email protected]> * examples: Modify verified version Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Generate polkadot.rs from `0.9.18-4542a603cc-aarch64-macos` Signed-off-by: Alexandru Vasile <[email protected]> * examples: Update polkadot_metadata.scale from `0.9.18-4542a603cc-aarch64-macos` Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Update documentation Signed-off-by: Alexandru Vasile <[email protected]> * tests: Modify default pallet usage Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Remove hex dependency Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Add MetadataTestType to capture complex types Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Update tests to use complex types Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Check metadata correctness via extending pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Extend pallet hash with Events Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Extend pallet hash with constants Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Extend pallet hash with error Signed-off-by: Alexandru Vasile <[email protected]> * examples: Extend metadata compatibiliy with StorageApi and ConstantsApi Signed-off-by: Alexandru Vasile <[email protected]> * Modify comments and documentation Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Benchmarks for full validation and pallet validation Signed-off-by: Alexandru Vasile <[email protected]> * metadata/benches: Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Hash metadata just by inspecting the provided pallets Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Make pallets generic over T for `AsRef<str>` Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Expose the name of the pallets composing the metadata Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Update polkadot.rs with pallets name Signed-off-by: Alexandru Vasile <[email protected]> * codegen: Obtain metadata hash only by inspecting pallets Signed-off-by: Alexandru Vasile <[email protected]> * codegen,subxt: Extend the metadata hash to utilize just pallets Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Update polkadot.rs with client metadata has per pallet Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Test `get_metadata_per_pallet_hash` correctness Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * metadata/benches: Fix decode of metadata Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * [static metadata] validate storage, calls and constants per call (paritytech#507) * validate storage, calls and constants per call * fix typo * cache per-thing hashing, move an Arc, remove some unused bits * create hash_cache to simplify metadata call/constant/storage caching * simplify/robustify the caching logic to help prevent mismatch between get and set * cargo fmt * Fix clippy Signed-off-by: Alexandru Vasile <[email protected]> * bench the per-call metadata functions * metadata: Add test for `node_template_runtime_variant` Signed-off-by: Alexandru Vasile <[email protected]> * ensure criteron cli opts work * group benchmarks and avoid unwrap issues * metadata: Check template runtime for handling the pallet swap order case Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Remove debug logs Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Optimise by removing field's name and type_name and type's path Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Refactor `get_type_hash` to break recursion earlier Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Add tests for `hash_cache` Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Add tests for checking Metadata Inner cache Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Check semantic changes inside enum and struct fields Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Add enums named differently with compatible semantic meaning Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Guard testing of release versions for `node_template_runtime` Signed-off-by: Alexandru Vasile <[email protected]> * Improve documentation Signed-off-by: Alexandru Vasile <[email protected]> * Update polkadot.rs Signed-off-by: Alexandru Vasile <[email protected]> * metadata/tests: Manually construct type of `node_template_runtimeL::Call` Signed-off-by: Alexandru Vasile <[email protected]> * no more special Call handling, avoid a little cloning, and actually sort by name * remove unused deps and fmt * RuntimeMetadataLastVersion => RuntimeMetadataV14 * remove a bunch of allocations in the metadata hashing, speed up from ~17ms to ~5ms * update release docs to release metadata crate too * subxt: Remove codegen dependency Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Replace std RwLock with parking_lot Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Add ws address to `TestNodeProcess` Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Add metadata validation integration test Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Allow setting metadata on the ClientBuilder Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Check incompatible metadatas Signed-off-by: Alexandru Vasile <[email protected]> * metadata: Fix constant hashing for deterministic output Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Check metadata validation for constants Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Test validation for calls Signed-off-by: Alexandru Vasile <[email protected]> * subxt/tests: Test validation for storage Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Expose `set_metadata` for testing only Signed-off-by: Alexandru Vasile <[email protected]> * subxt: Guard metadata tests under integration-tests Signed-off-by: Alexandru Vasile <[email protected]> Co-authored-by: James Wilson <[email protected]>
This PR introduces logic for checking signatures of functions imported by smart-contracts, providing convenient interface for these functions.
Since signature of a function and it's arguments are super related I defined
define_env
macro which purpose is two-fold:Don't be afraid of a big amount of add/removed diffs, it just
substrate_runtime_contract::vm
code was split on two modules:prepare
,env_def
.prepare
module handles loading, checking and preprocessing wasm modules) andenv_def
module contains the environment defintion (i.e. functions that can be imported by a contract) and all related machinery like macros.