Skip to content

Fix Linux wheel C++ runtime loading (#1566) - #1566

Closed
yutingye wants to merge 1 commit into
mainfrom
export-D110929696
Closed

yutingye wants to merge 1 commit into
mainfrom
export-D110929696

Conversation

@yutingye

@yutingye yutingye commented Jul 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

Linux PyPI wheels bundle a renamed private libstdc++ because conda-forge
runtime dependencies require newer GLIBCXX symbols than manylinux_2_28's
system libstdc++ provides.

The original import failure was caused by loading that private C++ runtime before
PyTorch under Python's default RTLD_LOCAL mode:

import pymomentum.geometry
import torch

That lets a second libstdc++ instance exist in the process before torch._C is
loaded, and torch can segfault during import.

A scoped RTLD_GLOBAL loader makes that order pass, but it is not a correct fix:
it exposes PyMomentum's private libstdc++ symbols to the whole process. In the
opposite order, that can poison later native library loads. In particular,
Triton's compiler fails in ConvertTritonGPUToLLVM after this order:

import torch
import pymomentum.geometry
import pymomentum.quaternion # backend="triton"

Fix this by changing the Linux wheel repair shim to import torch, when torch is
available, immediately before loading any PyMomentum native extension. The
extension still loads with Python's normal RTLD_LOCAL behavior, so PyMomentum's
private libstdc++ remains hidden from later native extensions such as Triton,
while torch owns the process-global C++ runtime state before PyMomentum's private
runtime is loaded.

The generated pymomentum/init.py is added only during Linux wheel repair.
This keeps macOS and Windows behavior unchanged for now because the known bug is
in Linux ELF/dlopen/libstdc++ interaction. If we later want one package shape on
all platforms, we can move a no-op-on-non-Linux init.py into the source tree,
but that should be a separate deliberate package-shape change.

Also expand wheel smoke tests so every wheel variant installs torch and checks
the pymomentum.geometry-before-torch order. The strict GPU wheel test continues
to require CUDA and Triton.

Reviewed By: cstollmeta

Differential Revision: D110929696

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 7, 2026
@meta-codesync

meta-codesync Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

@yutingye has exported this pull request. If you are a Meta employee, you can view the originating Diff in D110929696.

@meta-codesync meta-codesync Bot changed the title Fix Linux wheel C++ runtime loading Fix Linux wheel C++ runtime loading (#1566) Jul 7, 2026
meta-codesync Bot pushed a commit that referenced this pull request Jul 7, 2026
Summary:

When using pymomentum on linux, importing its submodules before importing torch will crash. 
```python
#import pymomentum.geometry
#import torch # crashes
```
It's because the wheel packaging step re-packages and renames stdlibc++, so importing torch next creates a duplicated c runtime. The reason we did it is because different runtime libs require different gcc versions... 

This diff adds pymomentum/__init__.py to handle this conflict for linux.

Differential Revision: D110929696
@meta-codesync
meta-codesync Bot force-pushed the export-D110929696 branch from 36a651c to 20bb32c Compare July 7, 2026 22:02
meta-codesync Bot pushed a commit that referenced this pull request Jul 8, 2026
Summary:

Linux PyPI wheels bundle a renamed private libstdc++ because conda-forge
runtime dependencies require newer GLIBCXX symbols than manylinux_2_28's
system libstdc++ provides.

The original import failure was caused by loading that private C++ runtime before
PyTorch under Python's default RTLD_LOCAL mode:

  import pymomentum.geometry
  import torch

That lets a second libstdc++ instance exist in the process before torch._C is
loaded, and torch can segfault during import.

A scoped RTLD_GLOBAL loader makes that order pass, but it is not a correct fix:
it exposes PyMomentum's private libstdc++ symbols to the whole process. In the
opposite order, that can poison later native library loads. In particular,
Triton's compiler fails in ConvertTritonGPUToLLVM after this order:

  import torch
  import pymomentum.geometry
  import pymomentum.quaternion  # backend=\"triton\"

Fix this by changing the Linux wheel repair shim to import torch, when torch is
available, immediately before loading any PyMomentum native extension. The
extension still loads with Python's normal RTLD_LOCAL behavior, so PyMomentum's
private libstdc++ remains hidden from later native extensions such as Triton,
while torch owns the process-global C++ runtime state before PyMomentum's private
runtime is loaded.

The generated pymomentum/__init__.py is added only during Linux wheel repair.
This keeps macOS and Windows behavior unchanged for now because the known bug is
in Linux ELF/dlopen/libstdc++ interaction. If we later want one package shape on
all platforms, we can move a no-op-on-non-Linux __init__.py into the source tree,
but that should be a separate deliberate package-shape change.

Also expand wheel smoke tests so every wheel variant installs torch and checks
the pymomentum.geometry-before-torch order. The strict GPU wheel test continues
to require CUDA and Triton.

Differential Revision: D110929696
@meta-codesync
meta-codesync Bot force-pushed the export-D110929696 branch from 20bb32c to f999266 Compare July 8, 2026 01:39
Summary:

Linux PyPI wheels bundle a renamed private libstdc++ because conda-forge
runtime dependencies require newer GLIBCXX symbols than manylinux_2_28's
system libstdc++ provides.

The original import failure was caused by loading that private C++ runtime before
PyTorch under Python's default RTLD_LOCAL mode:

  import pymomentum.geometry
  import torch

That lets a second libstdc++ instance exist in the process before torch._C is
loaded, and torch can segfault during import.

A scoped RTLD_GLOBAL loader makes that order pass, but it is not a correct fix:
it exposes PyMomentum's private libstdc++ symbols to the whole process. In the
opposite order, that can poison later native library loads. In particular,
Triton's compiler fails in ConvertTritonGPUToLLVM after this order:

  import torch
  import pymomentum.geometry
  import pymomentum.quaternion  # backend=\"triton\"

Fix this by changing the Linux wheel repair shim to import torch, when torch is
available, immediately before loading any PyMomentum native extension. The
extension still loads with Python's normal RTLD_LOCAL behavior, so PyMomentum's
private libstdc++ remains hidden from later native extensions such as Triton,
while torch owns the process-global C++ runtime state before PyMomentum's private
runtime is loaded.

The generated pymomentum/__init__.py is added only during Linux wheel repair.
This keeps macOS and Windows behavior unchanged for now because the known bug is
in Linux ELF/dlopen/libstdc++ interaction. If we later want one package shape on
all platforms, we can move a no-op-on-non-Linux __init__.py into the source tree,
but that should be a separate deliberate package-shape change.

Also expand wheel smoke tests so every wheel variant installs torch and checks
the pymomentum.geometry-before-torch order. The strict GPU wheel test continues
to require CUDA and Triton.

Reviewed By: cstollmeta

Differential Revision: D110929696
@meta-codesync
meta-codesync Bot force-pushed the export-D110929696 branch from f999266 to edf33e4 Compare July 14, 2026 00:10
@meta-codesync meta-codesync Bot closed this in 3b45e02 Jul 14, 2026
@meta-codesync

meta-codesync Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

This pull request has been merged in 3b45e02.

@meta-codesync meta-codesync Bot added the Merged label Jul 14, 2026
@yutingye
yutingye deleted the export-D110929696 branch July 14, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant