Skip to content

How to reconstruct your benchmarks? #1

@Robbepop

Description

@Robbepop

Hi, I am one of the authors of wasmi and got curious about your s1vm which claims to be significantly (approx 2.2 times) faster than wasmi for some fibonacci testcase.
I got seriously interested in your approach because of this claim and tried to reproduce your benchmarks.

My results so far with fib(25):

  • s1vm: 9.668629ms
  • wasmi_v0: 21.343 ms ~120% slower
  • wasmi_v1: 12.966 ms ~30% slower

Note that wasmi_v0 uses an old engine whereas wasmi_v1 is the most recent engine. Both use stack machine based bytecode which naturally is inferior with respect to performance to register machine bytecode used in both Wasm3 and s1vm. This paper indicates roughly 50%-150% performance improvement by switching over to a register machine base bytecode. So if wami_v1 were to use register machine based bytecode it would probably not be ~30% slower but ~35% faster than s1vm. In fact there is an issue open for wasmi to implement register machine based bytecode, however, we are not yet sure which instruction dispatching technique to use. For this we currently research into the most effective dispatching techniques and closure-based approaches so far do not seem to be very promising unfortunately.

To reconstruct my own benchmarks you can checkout wasmi using this branch. And a fork of your s1vm using this link.

For wasmi simply use cargo bench fib_recursive and for the s1vm fork use cargo run --release -- fib.wasm fib_recursive 30.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions