Isolating execution via separate contexts

[mashdragon]

Describe your feature request here.

It does not seem like there is a feature in PythonMonkey to run JavaScript scripts or evals in isolation after importing the module. This means that I have to keep track of the JavaScript state myself and perform any cleanup manually on objects that pollute the globalThis space.

It would be nice if there were a way to isolate execution so that separate scripts run in separate environments, just like how scripts in the web are isolated to their pages.

Maybe this would appear as a pm.context() function to get a new JavaScript execution context.

Code example

import pythonmonkey as pm

# Potential implementation of this context feature
context = pm.context()
context.eval("x = 4")

# A different context
context2 = pm.context()
context2.eval("x = 5")

print(context.eval("x")) # Prints 4
print(context2.eval("x")) # Prints 5

[wiwichips]

Hi @mashdragon, thanks for taking the time to check out PythonMonkey! This is definitely on our roadmap and something we will get to in the future

[hjdhnx]

I think it's a good idea, quickjs did it
see here

[wesgarland]

This is definitely an interesting idea.

It does raise some complex questions like - what happens when a JS variable is shared from one context to another via the underlying Python code? The GC requirements in PythonMonkey can get very interesting because we have to interoperate with both interpreters at the same time.

One of the questions that pops is -- do we care about complete isolation, meaning a new SpiderMonkey JSContext *cx, or is symbol-level isolation enough?

Providing a new global object and standard classes (Array, Object, etc) would be pretty easy, along with an isolated module context via ctx-module.

But this type of incomplete isolation might have interesting prototype-walking attacks, like [].constructor.prototype ... might be outside of the symbol isolation context because there is only one struct JSClass * per standard class in a JSContext *cx even when different instances of the standard classes are available in JavaScript.

I like the QuickJS interface. If we do this, we will also make it possible to emulate the NodeJS vm module calls somehow.

[wesgarland]

I think what we want is separate realms. We can call 'em contexts from the users' POV.

http://www.carolinecullen.com/spidermonkeynotes/notes/runtime.html

[mgberg]

I just ran into PythonMonkey, and so far it's been great! I just wanted to bump this issue because I also would find the ability to create separate contexts extremely useful.

[dsetzer]

This feature is definitely a must. I'm a little shocked that it's sitting on the backburner as far as I can tell so far the only way to achieve this would be through the use of IIFE's to run a set of eval's concurrently while keeping them each in their own context?

I haven't tried it yet though, so I'm not sure how well that would work with pm.wait() and pm.collect()

@wiwichips is there any updated on this or a generally accepted way of achieving this yet? or how would IIFE's work with pm.wait() and pm.collect() when having multiple concurrent evals running which are using eventemitters to control their states?

[wesgarland]

@dsetzer a completely isolated context is quite different from what you can do with IIFE symbol isolation. For instance, an object literal declared in one context is not an instance of Object in another context. Then we have to consider the issues like how to deal with JS objects that cross the JS cx boundary via PythonMonkey. Do we double-proxy? Re-wrap? Pass raw jsval pointers? How do Python reference counts work, etc. Does it make sense to have an event loop per context? etc. (JS generally has one loop per process, but async Python can get weird).

I saw in the notes from your closed PR that you were considering the SuperGlobal case. I love super globals and went out of my way to implement them in gpsee back in the day. But JS contexts don't share a global, the fact that contexts each have their own globalThis is one of their defining characteristics. (cx here is like a v8 isolate if you're used to v8? Think one cx = one browser tab). Maybe we need to think about terminology a bit?

If you wind up with a patch you'd like to upstream - we'd be interested. I, too, agree that this is a very important feature in the long term.