Support copying an Instance

I've encountered some proxy-wasm modules (https://github.com/corazawaf/coraza and https://higress.cn/en/docs/latest/plugins/ai/api-o11y/geo-ip/) which do large amount of initialization work. Since the modules are not thread safe, if I want to be able to concurrently use the module, I would need to create multiple instances of the module. But since the initialization is so heavy, this would really slowdown the system. It would be nice if you could initialize one module and then copy it as needed for each concurrent thread.

I think the biggest impact would be that the WasmFunctionHandle's likely need to know which one of the instances it's dealing with. So maybe an instance could provide something like a int id = instance.id() method so we can tell which is which.

@chirino

Thanks for opening this issue @chirino , in the past we have used Wizer to tackle this issue, and we can experiment with InstaBoot too.

I believe that the major issues are going to pop up when you start using the WASI layer (e.g. open files etc. etc.).

I'm happy to review a plan for what should be serialized and what can be restored before jumping into a PR.

Another option would be to force the user to reset all instance imports on on or after a copy.

Let's discuss further on #946

Before looking at an implementation I'd like to better understand what is the use case and tradeoffs with the considered alternatives.
More specifically:

But since the initialization is so heavy

This is arguably not accurate, or, at least, can be much much cheaper by using Wizer and similar.
The instance initialization is, per se(code wise), pretty fast and straightforward; or have you measured something different?

Giving the ability to copy an instance comes with a number of shortcomings and I miss an analysis of those, for example, initializers can depend on env variables of arguments, by using a copy people can break the behavior defined by the spec.

Take Coraza for example: if you use the following plugin configuration (which is read during init) then it takes a few seconds to start.

value: |
{
    "directives_map": {
        "default": [
            "Include @demo-conf",
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-conf",
            "Include @owasp_crs/*.conf"
        ]
    },
    "default_directives": "default"
}

That's because it's loading all the rules in owasp_crs/*.conf.

How do I use wizer on build time compiled module that will be changing what it does in the init phase based on runtime config?

FYI, proxy-wasm modules only have to implement like 3 functions of WASI.

That's because it's loading all the rules in owasp_crs/*.conf.

How the configuration is loaded? Is it using the WASI layer and reading from the disk? or something else?
Have you attempted to use projects like: https://github.com/kateinoigakukun/wasi-vfs ?

the init phase based on runtime config

This is suspicious, Wasm payloads are intended to boot fast, if there is some super heavy computation done in the initialization phase of this specific module I'd like to understand more about it before jumping on a solution.

All in all, I think the biggest gap is communication here, you are proposing a solution without an adequate context for the reader, I need to ask you to better detail the problem first:

what is the problem we are trying to solve?
what components are involved? (a very minimal reproducer would be ideal)
possible alternatives?
tradeoff of the proposed solution

No, proxy-wasm does not provide WASI file access.

I don't know exactly how it loads those file references, but I would assume the plugin has statically compiled/embedded a virtual FS into the wasm binary. This is something that that is part of the std library in go.

I don't understand why you would think proxy-wasm plugins can't be slow to init? I've got at least two of them that are. Are you saying there is a rule against that?

I've added a test that can help you see this issue in effect. Check out:
https://github.com/chirino/proxy-wasm-java-host/tree/slow-init

and then run the test I added:

mvn clean install -Dtest=SlowInitTest -pl proxy-wasm-java-host

For me, it outputs:

...
[INFO] Running io.roastedroot.proxywasm.examples.SlowInitTest
/fast-config.json: Host start() execution time: 123.415 ms
/slow-config.json: Host start() execution time: 3093.737 ms
..

It shows that module init speed can vary drastically based on the plugin configuration used for the plugin.

@evacchi

I don't understand why you would think proxy-wasm plugins can't be slow to init? I've got at least two of them that are. Are you saying there is a rule against that?

I do believe they might be slow, IIRC Go proxy-wasm plugins have been deprecated somewhere(Kong? maybe @evacchi remembers) due to memory and slowness issues.

What's the root cause analysis of the slowness?

I don't understand why it matters WHY the module is slow.

I can work around the modules slow issue by copying the instance after it starts.

unfortunately, I think this might be more delicate than it seems on a first sight;

e.g. say the init routine involves invoking some host functions; as you point out, those host functions, if shared, might at the very least need to know the instance they are being invoked by; but if those host functions are stateful or are doing some side effect (highly likely, otherwise you wouldn't need them to be computed on the host), then it might not be clear how to safely clone the state across instances.

OR, if there are imports of any other kind, the init routine might be affecting the state of a different module.

etc.

I think we briefly discussed this in the past and we ended up not doing it for these reasons :/

EDIT: see e.g. #464

In my case (proxy-wasm), I an in full control of all the imports provided to the guest module. I can also track all the functions called by the init function, so after module init is started, I'll know if it's safe to copy the instance or not.

FYI proxy-wasm init function typically get the plugin config and optionally do some logging. These actions don't affect the state of the host.

I don't understand why it matters WHY the module is slow.

It matters because I’m genuinely trying to help solve the actual problem, but I can’t do that without understanding it.

We’ve evaluated exposing a copy method on Instance before (see #464), and the outcome was clear:

Linking breaks.
WASI breaks.
Stateful host functions break.
Even simple shared state like a Random source causes unexpected behavior.
It opens the door to foot-gun territory for anyone using it.
It defies common expectations of what “copy” should do.
It’s unsupported by the WASM spec.
It would be a workaround for two specific modules, subject to accidental break in the future as we do not understand the real cause of the slowdown.
It will introduce bugs that are a nightmare to debug (race conditions, state leaks, etc.).

So unless there are compelling arguments to addresses all of these points, a copy() API on Instance in Chicory just isn’t going to happen.

I can work around the module's slow issue by copying the instance after it starts.

That might work for you, but it’s not a safe or sustainable solution for Chicory as a whole.
Just because a local hack works under specific circumstances doesn’t mean it should be promoted into the public API, especially one with so many known pitfalls.
Instance isn’t final, so you can subclass it and live with the risks in your own codebase.

Worst case scenario, when we understand the actual performance issue, and it turns out a general escape hatch is really needed, we can design a proper, sustainable API for it.
But that only happens after we understand the problem, not instead of it.

I'm here to help, and, in the story of this project, we’ve consistently seen that tackling the root issue leads to better outcomes than papering it over. Let’s aim for that.

I don't understand why it matters WHY the module is slow.

It matters because I’m genuinely trying to help solve the actual problem, but I can’t do that without understanding it.

Is that because your thinking you speed up the guest's init by optimizing the chicory engine?

We’ve evaluated exposing a copy method on Instance before (see #464), and the outcome was clear:

Linking breaks.

WASI breaks.

Stateful host functions break.

Even simple shared state like a Random source causes unexpected behavior.

It opens the door to foot-gun territory for anyone using it.

All these limitations can be documented.

It defies common expectations of what “copy” should do.

Lets call it something else then, open to sugestions.

It’s unsupported by the WASM spec.

It would be a workaround for two specific modules, subject to accidental break in the future as we do not understand the real cause of the slowdown.

When you say "we do not understand the real cause of the slowdown.", it seems you don't think it's something as simple as the module is doing lots computationally intensive work right?

It will introduce bugs that are a nightmare to debug (race conditions, state leaks, etc.).

These would be the problems of the host implementer. Give them the warnings.

So unless there are compelling arguments to addresses all of these points, a copy() API on Instance in Chicory just isn’t going to happen.

I'm sure it can all be worked out.

I can work around the module's slow issue by copying the instance after it starts.

That might work for you, but it’s not a safe or sustainable solution for Chicory as a whole. Just because a local hack works under specific circumstances doesn’t mean it should be promoted into the public API, especially one with so many known pitfalls. Instance isn’t final, so you can subclass it and live with the risks in your own codebase.

I would be fine with this. Keep in mind all instance fields and helper classes would need to be made accessible to the subclasses in a different package so, likely making lots of field protected and maybe opening up new classes. An alternative would be to make the copy method protected so that only subclasses can access this copy() functionality.

Worst case scenario, when we understand the actual performance issue, and it turns out a general escape hatch is really needed, we can design a proper, sustainable API for it. But that only happens after we understand the problem, not instead of it.

I'm here to help, and, in the story of this project, we’ve consistently seen that tackling the root issue leads to better outcomes than papering it over. Let’s aim for that.

I'll try wrap my head around the coraza plugin is doing and report back. It's likely just doing lots of computation of the rules that are being enabled in the plugin.

Quick analysis of what the coraza plugin is doing. When configured with

value: |
{
    "directives_map": {
        "default": [
            "Include @demo-conf",
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-conf",
            "Include @owasp_crs/*.conf"
        ]
    },
    "default_directives": "default"
}

It will parse all the directives in this directory: https://github.com/corazawaf/coraza-proxy-wasm/tree/main/wasmplugin/rules/crs
All those files get statically embeded into the go WASM binary (not accessed with WASI).

$ cat *.conf | wc
   12491   39891  596359

That's about over 12 thousand lines of configuration to parse. Further more that configuration may contain references to other data files which also need to be parsed and loaded into data structures:

cat *.data | wc
    8255   20593  190975

Given it's doing so much parsing and processing of data, the 2-3 second start up times seems reasonable.

I would be fine with this. Keep in mind all instance fields and helper classes would need to be made accessible to the subclasses in a different package so, likely making lots of field protected and maybe opening up new classes. An alternative would be to make the copy method protected so that only subclasses can access this copy() functionality.

I know it's not ideal but might be a middle ground.

All those files get statically embeded into the go WASM binary (not accessed with WASI).

From what I hear it should be possible to initialize(parse) a complete configuration, snapshot with wizer and use the produced module.
Have you attempted to go down this path? Have you found any blocker?

I would be fine with this. Keep in mind all instance fields and helper classes would need to be made accessible to the subclasses in a different package so, likely making lots of field protected and maybe opening up new classes. An alternative would be to make the copy method protected so that only subclasses can access this copy() functionality.

I know it's not ideal but might be a middle ground.

What do you think about new version in #946 I think it does decent job at hiding the copy method, for those that do find it, I think they will understand that it's dangerous to use.

All those files get statically embeded into the go WASM binary (not accessed with WASI).

From what I hear it should be possible to initialize(parse) a complete configuration, snapshot with wizer and use the produced module. Have you attempted to go down this path? Have you found any blocker?

I would not be able to do that in a native image and run compiled right?

I would not be able to do that in a native image and run compiled right?

It really depends on the setup and how Coraza works ...

Navigation Menu

Repository navigation

Support copying an Instance #945

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support copying an Instance #945

Description

Activity

chirino commented on Jun 11, 2025

andreaTP commented on Jun 11, 2025

chirino commented on Jun 11, 2025

chirino commented on Jun 13, 2025

andreaTP commented on Jun 16, 2025

chirino commented on Jun 16, 2025

chirino commented on Jun 16, 2025

andreaTP commented on Jun 17, 2025

chirino commented on Jun 17, 2025

chirino commented on Jun 17, 2025

andreaTP commented on Jun 17, 2025

chirino commented on Jun 17, 2025

evacchi commented on Jun 17, 2025

chirino commented on Jun 17, 2025

andreaTP commented on Jun 18, 2025

chirino commented on Jun 18, 2025

chirino commented on Jun 18, 2025

andreaTP commented on Jun 18, 2025

chirino commented on Jun 18, 2025

andreaTP commented on Jun 23, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions