Add IECORE_RTLD_GLOBAL environment variable #1127
Conversation
|
I would need the 2 commits from my other PR affecting IECoreMaya & IECoreHoudini as well. Do you want to cherry-pick those over (and fixup to use the new env var name) or should I try pushing into your branch myself? Or alternatively I could cherry-pick your commits into my PR. I don't recall why we didn't merge mine back then, but I'd guess it was about not wanting the |
|
I've pushed 9654922 which fixes the last problem running Gaffer with
My aim in this PR is to get the minimum requirement for Gaffer standalone merged (while not changing default behaviour), and then hand over to you to take on the broader DCC/IE stuff as a followup. From the comments on #954 it seems there might be unresolved problems with Maya/Houdini even with those commits, so I'd rather not cherry-pick blindly until those are addressed. |
|
Would be good to get this merged so I can follow up with the matching PR in Gaffer (targeting 0.60). As mentioned above, this shouldn't change anything (except 9654922) unless opted into, so should be safe for the custom IE builds. |
This is only an alpha release in the sense that we will probably bump major version for some other dependencies (e.g. USD) before calling it 3.0.0. All the dependencies are from officially released (non-alpha) versions. The intention is to use 3.0.0 for Gaffer 0.60, but 3.0.0 is also a better approximation of the environment used for deployment at Image Engine.
This can be used to turn off the use of the `RTLD_GLOBAL` flag in Python, which makes us less vulnerable to symbol clashes between libraries. Updated CI to run the unit tests with it turned off.
This was causing crashes at render shutdown in Gaffer unless we either used `RTLD_GLOBAL` or we used `LD_PRELOAD=libai.so`. The crashes were always during deallocation, so I suspect that we were getting mismatched allocation/deallocation between Arnold's allocator and our own.
|
I'm testing this at IE now. Do you have an accompanying Gaffer PR? So far Cortex tests are all passing with RTLD_GLOBAL disabled, but I can't do interactive testing because libIERendering complains about |
andrewkaufman
force-pushed
the
dlopenFlagsPR
branch
from
9654922 to
73d3136
Compare
|
I just pushed the extra commits from my other PR necessary for testing this at IE... hope that's ok... |
|
To clarify, these extra commits are good-to-go on our end. The branch works fine in Maya/Houdini/Gaffer when So for us its ok to merge the PR, but it would be nice to confirm the Arnold change when |
|
Looks like the py3 test failure is that I rebased on master. I'm not really sure why what test failure didn't showup on #1128 directly though... maybe I accidentally merged that one before the CI actually started... |
I don't have a PR, and won't until I update the dependencies package once this Cortex PR is merged. But this is the branch I've been testing with : https://github.com/johnhaddon/gaffer/tree/rtldGlobalDisabling There are no changes really, just something in the wrapper to set
At first glance that seems to be exported correctly in Gaffer. Perhaps you need to link against
I'll open a PR to fix that. |
This adds an environment variable which can be used to turn off the
RTLD_GLOBALflag we've used to load our python modules up until now. After an unpleasant day or two of debugging where this was implicated but turned out not to be the issue, I'd really like to make some progress towards to getting rid of it so there's one less thing to worry about. @andrewkaufman, having done the work, I now realise you have something similar already in #954. The difference here is that I've also updated the CI setup to run all the unit tests with the flag turned off, demonstrating (hopefully) that all is well.The dependencies update here isn't purely coincidental - it seemed that older versions of
pyopenvdbwould crash on load if the flag was off, independent of our code, but that appears to be fixed in 7.2.2. The next step is to update Gaffer to turn the flag off by default, and I'm getting close with that - everything working so far except for an issue with the Arnold display driver.