Skip to content

[JITLink][AArch32] Add explicit visibility macros to functions needed by unittests #116557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fsfod
Copy link
Contributor

@fsfod fsfod commented Nov 17, 2024

Without these there will be missing symbol errors when building JITLinkTests for windows shared library builds with explicit symbol visibility macros are enabled.

This is part of the work to enable LLVM_BUILD_LLVM_DYLIB and LLVM plugins on window.

@weliveindetail
Copy link
Member

Thanks for bringing this up. These functions were never meant to be exported publically, we only want to access them for testing. Does that make a difference for the LLVM_ABI tag? Do we have similar situations anywhere else in the codebase?

In order to avoid confusion, we could also make a local LLVM_TEST_ABI redefinition with with a note. Or if this only affects Windows, we could disable the respective unittets there. What do you think?

@fsfod
Copy link
Contributor Author

fsfod commented Nov 24, 2024

Thanks for bringing this up. These functions were never meant to be exported publically, we only want to access them for testing. Does that make a difference for the LLVM_ABI tag? Do we have similar situations anywhere else in the codebase?

Because these are not declared in any header there wouldn't really be any expectation that these could be consumed by anything but tests.
Yes i've had to fix missing exports of internal symbols used by tests in other places of the LLVM codebase, sometimes its been internal headers which are easier to automate annotations.

In order to avoid confusion, we could also make a local LLVM_TEST_ABI redefinition with with a note

Maybe i could, but previous discussions i've had with @compnerd he wanted to avoid proliferation of lots of different symbol visibility macros.

if this only affects Windows, we could disable the respective unittets there. What do you think?

Its only affects windows atm, but in the future when default symbol visibility is changed changed to hidden for the LLVM shared library these would need the same annotations as windows

@compnerd
Copy link
Member

Maybe i could, but previous discussions i've had with @compnerd he wanted to avoid proliferation of lots of different symbol visibility macros.

We will need a macro for each shared library that we support. If we add more macros for each library, it would become a huge number and can be overwhelming for other developers not familiar with this.

Copy link
Member

@weliveindetail weliveindetail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so let's keep it simple. LGTM

@fsfod fsfod force-pushed the exported-api/arch32-jit-exports branch from f10915e to 8a51f64 Compare November 25, 2024 15:57
@lhames
Copy link
Contributor

lhames commented Dec 12, 2024

@compnerd Couldn't we have one global LLVM_TEST_ABI macro? I think it's fine to claim that we're either building llvm-project (all libraries) for testing or we're not.

I wouldn't want these symbols leaking into the ABI of release builds.

@compnerd
Copy link
Member

@lhames it seems reasonable at first glance, but quickly breaks down.

Let's say that we are building LLVMSupport as a DLL and LLVM statically. However, because LLVM_TEST_ABI is defined once and doesn't consider the target's build type into consideration, we end up with:

#if defined(LLVMSupport_STATIC)
#define LLVM_TEST_ABI /**/
#else
#if defined(LLVMSupport_EXPORTS)
#define LLVM_TEST_ABI __declspec(dllexport)
#else
#define LLVM_TEST_ABI __declspec(dllimport)
#endif
#endif

Now, when we build LLVM as a static library, LLVMSupport_STATIC is not defined (because it was built dynamically), and because LLVMSupport is not being built here, LLVMSupport_EXPORTS is undefined. So you end up with LLVM_TEST_ABI as __declspec(dllimport), which says that the definitions that LLVM is exporting for use in test are defined elsewhere.

Because the behaviour of the *_ABI macro is dependent on the module being built and how that module is being built, we cannot easily just use a single definition.

Now, maybe, we can get LLVM down to the small enough surface that we can have only 2 libraries (LLVMSupport and LLVM where the second one has ALL architectures enabled and is sufficient to be used for all tools and still is well under 64K entry points including data as part of the ABI). In such a case, yes, it seems reasonable to have two test ABI macros.

My concern is that we might end up having to extract each architecture backend into a separate DLL and then have the optimizations be pulled out, the assembler pulled out, and suddenly you have a set of test ABI macros.

@lhames
Copy link
Contributor

lhames commented Dec 12, 2024

@compnerd Can you dllexport and then dllimport the same symbol within a static link? (Apologies for my lack of windows knowledge here). In that case I'm imagining that we could have something as simple as:

#ifndef NDEBUG
#define LLVM_UNITTEST_EXPORT __declspec(dllexport)
#else
#define LLVM_UNITTEST_EXPORT
#endif

Then wrap the unit test in #ifndef NDEBUG and unconditionally dllimport the decl?

@compnerd
Copy link
Member

The declaration and definition must match (that is an error). So the declaration must be __declspec(dllexport) if there is a definition. You cannot have a declaration that is __declspec(dllimport) and then a definition.

That said, if you incorrectly mark the symbol as dllimport and then define it elsewhere within the module, that is generally a linker warning (but it will try to create a thunk to workaround it).

Copy link
Contributor

@lhames lhames left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@compnerd Ok. Sounds like finer grained export / import is impractical. Thanks very much for taking the time to talk that through with me!

@fsfod fsfod force-pushed the exported-api/arch32-jit-exports branch from 8a51f64 to caf4427 Compare January 30, 2025 01:56
@weliveindetail
Copy link
Member

@fsfod Should we land this?

… by unittests

Without these there will be missing symbol errors when building JITLinkTests for
windows shared library builds with explicit symbol visibility macros are enabled.

This is part of the work to enable LLVM_BUILD_LLVM_DYLIB and LLVM plugins on window.
@fsfod fsfod force-pushed the exported-api/arch32-jit-exports branch from caf4427 to 2caf6aa Compare April 20, 2025 16:59
@fsfod
Copy link
Contributor Author

fsfod commented Apr 20, 2025

I think so it depends on if @andrurogerz needs it for his work on annotating exports as well.

@andrurogerz
Copy link
Contributor

I think so it depends on if @andrurogerz needs it for his work on annotating exports as well.

I had to make these same patches locally to build the test against an LLVM DLL, so this PR is still relevant. If you're willing/able to merge it that'd be wonderful.

@fsfod
Copy link
Contributor Author

fsfod commented Apr 22, 2025

I don't have commit access so could someone merge it for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants