Import a memory object

[geloizi]

Let's say wasm runtime exports an additional memory object. How is it possible to import it? I've tried:

@external("env", "mymem")
declare var mymem: usize;

This doesn't work :(

[sampaioletti]

Try compiling with the --importMemory flag i.e.

asc hello.ts --importMemory

See More info in the book:
https://docs.assemblyscript.org/details/compiler

[geloizi]

If I add --importMemory it cannot find the "main" memory symbol

LinkError([ImportNotFound { namespace: \"env\", name: \"memory\" }])

The idea is to have an additional memory object shared between runtime and wasm instance.

[sampaioletti]

Can you share some code, what runtime (host) are you using? If you do --importMemory the host must provide a memory instance for the module to import .. That's what error your getting. So you either have to provide the memory in the host for use in the wasm..or grab the exported memory from the wasm and use it in the host.. I'll provide an example when I have a minute.. But if I'm missing your question please help me understand and I'll see if I can help you out. Sam

…

________________________________ From: geloizi <[email protected]> Sent: Saturday, October 5, 2019 7:03:41 PM To: AssemblyScript/assemblyscript <[email protected]> Cc: sam <[email protected]>; Comment <[email protected]> Subject: Re: [AssemblyScript/assemblyscript] Import a memory object (#886) If I add --importMemory it cannot find the "main" memory symbol LinkError([ImportNotFound { namespace: \"env\", name: \"memory\" }]) The idea is to have an additional memory object shared between runtime and wasm instance. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#886?email_source=notifications&email_token=ACDGQ52ZSHP735G266DHKATQNE2O3A5CNFSM4I5YSPZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAN7DNA#issuecomment-538702260>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ACDGQ5635SQ4EZ44V43KITTQNE2O3ANCNFSM4I5YSPZA>.

[geloizi]

I understand the problem a little better now. My AS code tries to import "shared" memory using next declaration:

@external("ns", "mem")
declare var mem: usize;

The compiler produces:

(import "ns" "mem" (global (;0;) (mut i32)))

Obviously this is incorrect because type of the import has to be memory and not global. What is the right way to import additional memory? Is it possible at all?

[sampaioletti]

Sorry yeah I can help with that...you don't import memory that way.

Forgive me if I go into too much detail or cover something you know...i'm just trying to help. And i won't claim to be an expert in wasm/AS...do if someone corrects me i'd trust them first (:

So I'll give my best not knowing your exact scenario.

Memory in wasm is a linear buffer or a linear chunk of continuous memory with a starting address and a length, by default wasm pages are 64Kib per 'page' where your module requests the number of pages when

In AS you don't manage the memory (unless you need to) It includes a run-time by default (you can build without it using different --runtime {full|half|stub|none}). Reason I'm covering that is because the memory isn't really for you...its for the runtime, and then AS builds instructions for the run-time to interact with the memory on your behalf (assuming you use the run-time)

So first about your question lets start with exporting, the default

So given the quickstart application from the book https://docs.assemblyscript.org/quick-start and i'll use node.js for my host environment.

and append the following line to the file

const foo="bar";

if we then build the module with

npm run asbuild:untouched

we get the following lines the the untouched.wat file (not in this order)

(memory $0 1) //tells host to create 1 page of memory and provide it to the module
(data (i32.const 256) "\06\00\00\00\01\00\00\00\01\00\00\00\06\00\00\00b\00a\00r\00")//the mem initializer with bar at 276
(global $assembly/index/foo i32 (i32.const 272)) //foo is located at index 272 in the mem
(export "memory" (memory $0)) //exports the memory instance back out to the host for use locally or another wasm module

So if we modify the index.js file to the following we can see the impact of that

const fs = require("fs");
const compiled = new WebAssembly.Module(fs.readFileSync(__dirname + "/build/untouched.wasm"));
const imports = {
  env: {
    abort(_msg, _file, line, column) {
       console.error("abort called at index.ts:" + line + ":" + column);
    }
  }
};
let exp=new WebAssembly.Instance(compiled, imports).exports
let mem=new Uint16Array(exp.memory.buffer,272,3)
let s=String.fromCharCode(...mem)
console.log("foo:",s)

> node index.js
> foo: bar

so that is how you would deal with an exported memory that you could then work with from the host and the wasm module

Importing memory

So now if we want to import memory we only have to do a couple of changes

rebuild with

npm run asbuild:untouched -- --importMemory

now our untouched.wat is identical with one exception

(import "env" "memory" (memory $0 1)) //import rather than declare

Meaning we have to provide that upon instantiation. So we modify our index.js slightly to provide that (or in your case, perhaps the host already does this)

const fs = require("fs");
const compiled = new WebAssembly.Module(fs.readFileSync(__dirname + "/build/untouched.wasm"));
const imports = {
  env: {
    abort(_msg, _file, line, column) {
       console.error("abort called at index.ts:" + line + ":" + column);
    },
    memory: new WebAssembly.Memory({initial:1})
  }
};
let exp=new WebAssembly.Instance(compiled, imports).exports
let mem=new Uint16Array(exp.memory.buffer,272,3)
let s=String.fromCharCode(...mem)
console.log("foo:",s)

results in the same

> node index.js
> foo: bar

now if you are implementing the host, you are free to do whatever you want with that memory before/after and then you can access it from the wasm module by creating an array buffer at the memory you set from the host...or whatever you would like

as a small note from a host standpoint the env.memory I used is Naive, normally you would read the imports from the module...but i haven't done enough with the webassembly browser api to know how you get that. Most of my experience is from embedded systems.

Hope that helps, i through up a repo with the example in it maybe i'll clean it up and PR it into the examples folder...someday (:
https://github.com/sampaioletti/assemblyscript-memory-example

Let me know if you need anything else.

[sampaioletti]

Sorry and to be clear you don't need the export declaration at all

// @external("env", "mymem")
// declare var mymem: usize;

you are correct you would use that if you were importing variables, functions etc, but memory is lower level than those constructs and is handled as I showed in the example.

[geloizi]

@sampaioletti Thanks a lot for your explanation! Indeed it's easy to access wasm instance memory from the host (exp.memory.buffer in your example).But my scenario is a little different. I'd like to share a memory region (let's say a single 64K page) between multiple wasm instances. So I thought about creating an additional memory object and exporting it into multiple wasm instances. I.e. each instance still has its own imported memory, but additionally some other memory with is mapped into other instances. Does this make sense?

[sampaioletti]

Yeah makes total sense, I've actually been trying something similar. See #887 and my fork at https://github.com/sampaioletti/assemblyscript/tree/shared-rt/examples/shared-rt.

There are a couple problems. The wasm mvp spec restricts us to one memory per module (someday I'm sure that will change) so we could write in support for multiple in AS but it wouldn't play well with anything else.

So you're down to three solutions share a single memory between all of them, copy values and alloc in the destination module when changing contexts, or some form of synchronization like channels our event emitters etc to exchange data. There is also some work in another PR supporting the future Thread spec which covers some shared memory and atomics but I haven't gotten into it enough to know how it will work.

From my three solutions our repo looks at the single shared memory solution. But you also have to share the runtime/gc or compile without the runtime. So I was playing with a shared runtime pattern. It still has some caveats related to memory initialization but it might give you some ideas or you could help me understand your use case and I can work it into what I'm doing.

[sampaioletti]

But if you want to play around with it just use the example I gave you before and create the memory region in a variable before you instantiate and pass it into multiple modules, I've done it and it works, but w/gc it eventually crashes as you have competing gcs managing the same space. Or you can build it without gc and have to manually manage it

[sampaioletti]

@geloizi not sure if your still playing with this but I put up another example as a result of our trial and error to see which pattern suited us best

I made a (naive) transform that allows you to pretty easily pass strings between modules. it required a couple of host functions added to the import env (see index.ts)

Thought it might give you some ideas. Good luck!

https://github.com/sampaioletti/assemblyscript/tree/rem-allocate/examples/rem-allocate

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.