Issues with 'static bounds on globals etc.

[ryankurte]

Hey hi, thanks for making a neat thing!

I've been playing with trying to pass references in to the lua environment and running up against the 'static bound limitations, and wondering whether there's a reason they are required, whether there's a convenient workaround i am missing, if they could be relaxed to 'lua to let the compiler manage them appropriately (which seems to me to be viable?).

As a super simple example, you can't really use references because &a is not 'static

struct Test (u32);

impl UserData for &Test {}

fn test(a: &Test) {
    // Create lua env
    let lua = Lua::new();
    
    lua.context(|ctx| {
        let globals = ctx.globals();
        globals.set("a", a);
    });
}

As far as I can see, so long as the lifetime of a exceeds 'lua this should be safe, as lua is known to drop before a? And if not (as it is currently), it'd be neat to have some documentation as to why and how to safely get around this limit with dynamic objects (i guess being able to add globals using scopes might be viable?).

This becomes even more complex when you have types with internal references, the only solution I have found is to transmute from local to 'static lifetimes (let t = std::mem::transmute::<Test<'a>, Test<'static>>(t)) on the basis that the object is known to live longer than the lua lifetime, however, if relaxing the bound to 'lua is unsound then this approach would also be.

[kyren]

The 'lua lifetime you're referring to is an invariant lifetime produced by the Lua::context call, you could cause memory unsafety by passing in a reference to a value that lives longer than the context call but shorter than Lua, then drop the a.

What you're asking for already exists as part of the scope system, it's just arguably less ergonomic in the situation where the parent Lua is only alive for as long as the inner scope call anyway. There could be an API to do this more simply that provides both a context and a scope object as part of a single callback, rather than making the user nest a scope callback inside a context callback, but the less ergonomic API to do what you want is available right now.

We tried to make this easier in the era before Context by bounding everything by 'lua instead of 'static, but that was unsound and we backed off of it (I had a much poorer understanding of the borrow checker back then, the design of rlua had all kinds of other problems too but that is actually OBVIOUSLY unsound to me now, just so everyone is aware). You would need a way of expressing that the 'lua lifetime is not a lifetime where the parameters and the Lua are both alive, but that the parameters must strictly outlive the Lua, and I couldn't find a way to soundly get the "strictly outlive" relationship.

Since then, the situation has actually gotten more complicated. The lifetimes in rlua are all actually convenient lies to get around the current lack of ATCs in the compiler, specifically to do with the internal Callback type. Currently as far as I know the lifetimes in rlua ARE sound, but they are DEFINITELY a hack, and a delicate hack at that. Adding the Context system was part of making that hack finally (hopefully) actually sound.

The lua globals table doesn't factor into it, once a value is inside Lua everything has an indefinite lifetime, you can take values you create from a Scope and put them absolutely anywhere inside Lua, the only issue is that when the scope ends the values become "invalidated" (every userdata method will return an error, function calls return an error). There's no way in Lua from even stopping this from occurring, which is why scope has to invalidate values in the first place.

This becomes even more complex when you have types with internal references, the only solution I have found is to transmute from local to 'static lifetimes (let t = std::mem::transmute::<Test<'a>, Test<'static>>(t)) on the basis that the object is known to live longer than the lua lifetime, however, if relaxing the bound to 'lua is unsound then this approach would also be.

It might be safe (as in, not exhibit UB) for you if you make sure that the t lives longer than lua, but it would be unsound for rlua to provide the ability to do that safely, since you could write safe code that uses it that exhibits UB.

[ryankurte]

Thanks for the reply!

The 'lua lifetime you're referring to is an invariant lifetime produced by the Lua::context call, you could cause memory unsafety by passing in a reference to a value that lives longer than the context call but shorter than Lua, then drop the a.

Ahh that's a misinterpretation on my part, I was aiming for for the lifetime of the Lua object rather than for the lifetime of the context object, I hadn't considered the strictly outlive difficulty, that's the interesting one.

What you're asking for already exists as part of the scope system, it's just arguably less ergonomic in the situation where the parent Lua is only alive for as long as the inner scope call anyway.

The lua globals table doesn't factor into it, once a value is inside Lua everything has an indefinite lifetime, you can take values you create from a Scope and put them absolutely anywhere inside Lua

This was enough of a clue that I worked out that you have to feed the scope method output into the globals methods, so the following totally works:

fn test(a: &Test) {
    // Create lua env
    let lua = Lua::new();
    
    lua.context(|ctx| {
        ctx.scope(|scope| {
            let globals = ctx.globals();
            scope.create_nonstatic_userdata(a);
        });
    });
}

Which is pretty much exactly what I wanted, but I hadn't worked this out from the docs / guided tour. I'd be happy to open a PR with some doctest examples on the scope methods to make their use more obvious if you're interested?

It might be safe (as in, not exhibit UB) for you if you make sure that the t lives longer than lua

Oh yes this was a horrendous fix to a problem that I couldn't see a way around, I'm glad there is an actual mechanism that I had just not understood, and thanks for your help!

[kyren]

Which is pretty much exactly what I wanted, but I hadn't worked this out from the docs / guided tour. I'd be happy to open a PR with some doctest examples on the scope methods to make their use more obvious if you're interested?

Yeah sure, that would be helpful! There's a very small bit about it in the docs for Context::scope here: https://docs.rs/rlua/0.17.0/rlua/struct.Context.html#method.scope but it's pretty subtle and it could use more documentation.

(Edit: That documentation is also confusing, because there are too many definitions of the word "lifetime" there)

[kyren]

This was enough of a clue that I worked out that you have to feed the scope method output into the globals methods, so the following totally works:

And yeah, you figured it out obviously, but I probably should have started by actually just showing you that you could do that rather than talking about it abstractly, sorry for not being more clear!