make the main zig executable no longer depend on LLVM, LLD, and Clang libraries

[andrewrk]

This issue is to fully eliminate LLVM, Clang, and LLD libraries from the Zig project. The remaining ties to these projects are as follows:

• completely eliminate dependency on LLD #8726
• LLVM
  • directly output LLVM bitcode rather than using LLVM's IRBuilder API #13265
  • C backend - 1742/1792 tests passing (97%)
  • enable the x86 backend by default for debug builds on x86_64-linux #22257
  • wasm backend - 1611/1765 tests passing (91%)
  • 100% behavior tests passing for the aarch64 backend #21172
  • ability to create import libs from def files without LLVM #17807
  • Avoid LLVM API for setting a module's code model and PIC/PIE levels #21238
  • completely eliminate dependency on LLVM library API calls #25492
• Clang
  • C++ source files in zig's repository are built by clang when bootstrapping
    • src/windows_sdk.cpp: port to Zig #15657
  • zig cc, zig c++, zig translate-c and other subcommands without a clang/llvm dependency in the compiler binary #20875
  • make resinator use aro's preprocessor instead of clang #17752
  • make mingw .def.in file parsing use aro's preprocessor instead of clang #17753
  • move @cImport to the build system #20630
• zig ar: a drop-in llvm-ar replacement #9828

Note that while the LLVM backend already outputs .bc files (#13265), the Zig compiler will lack the capability to compile .bc files into object files. Clang will need to be installed separately to handle this use case.

In exchange, Zig gains these benefits:

• All our bugs are belong to us.
• The compiler becomes trivial to build from source and to bootstrap with only a C compiler on the host system.
• We stop dealing with annoying problems introduced by Linux distributions and package managers such as Homebrew related to LLVM, Clang, and LLD. There have been and continue to be many.
• The Zig compiler binary goes from about 150 MiB to 5 MiB.
• Compilation speed is increased by orders of magnitude.
• We can implement our own optimization passes that push the state of the art of computing forward.
• We can attract research projects such as alive2
• We can attract direct contributions from Intel, ARM, RISC-V chip manufacturers, etc., who have a vested interest in making our machine code better on their CPUs.

Please read my other comments in this issue before replying:

• make the main zig executable no longer depend on LLVM, LLD, and Clang libraries #16270 (comment)
• make the main zig executable no longer depend on LLVM, LLD, and Clang libraries #16270 (comment)

[xdBronch]

I see the 1.0.0 milestone for this, is it meant to be a blocker for 1.0? or simply the goal

[andrewrk]

The milestone, on an issue labeled "proposal", means that a decision must be made to accept or reject that proposal before tagging the release corresponding to that milestone.

For an issue labeled "accepted", the milestone means that it must be implemented by then. So, if this proposal is accepted, then I will evaluate at that time which milestone to move it to.

[Jarred-Sumner]

In the near term, the machine code generated by Zig will become less competitive. Long-term, it may catch up or even surpass LLVM and GCC.

IMO, this is the biggest question. One of the most compelling reasons to use Zig is runtime performance of software written in Zig. Without LLVM's optimization passes, what will that look like?

[nektro]

Long-term, it may catch up or even surpass LLVM and GCC.

We can implement our own optimization passes that push the state of the art of computing forward.
We can attract research projects such as alive2
We can attract direct contributions from Intel, ARM, RISC-V chip manufacturers, etc., who have a vested interest in making our machine code better on their CPUs.

Zig will continue to implement optimization passes of its own over time and get faster.

[mlarouche]

So here the projects that depend on the ability to compile C++ that I currently developing:

• https://github.com/Cold-Bytes-Games/wwise-zig: I am using the C++ ability to compile glue code to create a C binding. I plan to use this for audio in my game BioMech Catalyst written in Zig.
• https://github.com/Cold-Bytes-Games/wwise-zig-demo: Using previous mentioned library plus zgui that does a nice Zig binding on top of Dear IMGUI
• Still planning to use zgui for my game editor in the future.

And also some NDA game platform that have C++ only API that will require some C++-to-C glue code to be compiled, but obviously not implemented.

To me, the ability to seamlessly build any C, C++ and Obj-C is a big selling point of the Zig toolchain even if it is behind a optional flag to enable LLVM and Clang when compiling Zig. A part of the hype momentum around Zig is due to that fact.

If this happens, I think I will remove my donation to the Zig Software Foundation.

TL;DR: Lots of libraries in the game development world (closed or open source) require the ability to compile C++.

[ikskuh]

I think this proposal would hurt the Zig ecosystem more than it would help it, due to several reasons:

• Adoption of compilers and languages in the embedded is basically driven by the code generation quality in terms of size. We have to be on-par or better than GCC and LLVM to be even considered tmto be adopted. 2%-5% bigger code is often not an annoyance, but a technical problem.
• A lot of projects use C++ code and might benefit from build.zig "as is". This could lead to adoption of Zig, with the build system as a kick-start. Removing support for C++ will hurt adoption a lot, because now the build of C++ projects with some additional Zig code will be even worse than just adding more cmake. Thus, a lot of projects wouldnt benefit from Zig in terms of quality of life, as it is yet another build tool.
• Zig is with its "batteries included" cross-compilation the best toolchain for doing native development. We would basically downgrade our toolchain to something Go where we rely on external tools to successfully build more complex projects. Even in the current state, its often so much easier to yeet zig at something to get it to build than even trying to get a cross-build running with existing tools
• Considering the scenario i have at work, introducing Zig as a build environment for several million lines of C++ would make a lot of people happy, as the builds would be faster, easier maintainable and trivially portable to other OS. With C++ and LLVM support removed, i dont see any chance of Zig adoption at this company

Imho, this proposal strongly violates the

Together we serve end users

idea, as the current direction we're heading is a really good unified native build environment based on a single static executable that can serve projects in arbitrar, sizes, shipping compilers for several major languages, a huge ass support for targets and a build system, making work in systems programming fun, even if one doesnt use Zig as a language.

We are on a good way to replace a huge list of tools with a single executable, making contributions to projects build on Zig fun, easy and platform independent.

When this proposal is accepted, in addition to Zig one will need to have the following tools installed:

• (GNU) Make
• premake
• meson
• CMake
• ninja
• $(arch)-$(os)-$(abi)-gcc
• llvm/clang
• GNU autotools
• m4
• vcpkg
• gradle
• conan
• qmake
• SCons
• maven
• ...

We can potentially replace all of those tools wit a single, equally powerful executable, making the live easier for all native devs out there

personal projects that would be affected:

• https://github.com/MasterQ32/cg-workbench (fully written in C++)
• https://github.com/MasterQ32/zero-graphics (Scintilla is written in C++)
• https://github.com/MasterQ32/zig-assimp (Assimp is the de-facto standard solution for generic 3D model loading, written in C++)

[foxnne]

I'm still a beginner I believe, but if it is at all worth it for me to give my point of view, Zig's ability to replace all of the build tools mentioned above is a big reason I was interested in Zig in the first place. I struggled a lot with all the different build systems and Zig is a really refreshing breath of fresh air.

I have two personal projects that use zgui heavily and I had planned on continuing.

• Pixi
• Aftersun

[emidoots]

With Mach engine we have two dependencies that would be very, very painful to remove (would set us back years):

1. DirectXShaderCompiler (DXC) - a Microsoft fork of LLVM 3.7, which generates DXIR (LLVM IR) which is what direct3d graphics drivers consume. Microsoft is trying to upstream support for emitting DXIR (specifically LLVM v3.7 IR) to the latest LLVM/clang version.
2. Calling into Apple's Metal shader compiler - which converts Metal's text shading language into (yet another fork) LLVM IR bytecode, which is what the Metal API consumes - only this one is proprietary and undocumented. Obj-C is the only way to invoke it, I believe.

It will be a long time before Zig's SPIRV backend is capable enough to generate non-GPGPU shaders for graphics APIs (if ever, since it would likely require major language changes) - so I don't see a way for us to escape these aside from replicating what these two projects -- LLVM forks -- do on our side.

Every other C++ dependency I believe we could safely escape from.

[emidoots]

A positive long-term effect of this change is that it would push us as a community away from wrapping C++ code and towards more pure-Zig solutions.

Many of the comments in this thread are about people using zgui, wwise, zig-gamedev, assimp, and other C++ libraries wrapped with Zig. It gives you a leg up in the short term, but I worry in the long-term that people's gamedev experiences coming to Zig will be 'initially I saw a nice language... then I encountered the guts of the libraries I was told to use were large, clunky C++ codebases'

[GeffDev]

Also a beginner, but I think being able to use Zig as a C++ build system (which is what I use for all my private C++ projects) is an invaluable feature to me and I believe many other people. The simplicity of Zig, having a compiler and a build system contained within a single executable (with the added bonus of being easily cross platform), is really cool, and it would be kinda unfortunate to see that feature be removed as an effect of removing clang and friends. However, if this does go through, the ability to fallback to generating .bc files and invoking clang is nice.

One of my friends pointed out that nothing stops you from invoking clang in a build.zig to compile C++ dependencies, even if Zig stops including clang. I wonder how much of a problem that would really be for these projects?

[xdBronch]

doing that would be an additional dependency without the ease of zigs cross compilation

Hmm, the more I think about this proposal the less I like it. I'm feeling it would be better to make the LLVM backend non-default (that is, switch -fno-LLVM for -fLLVM or something) but not remove it. It seems like this would solve most of the issues: the existence of many bugs that are LLVM's fault, and the slow compile speed, aren't strong reasons to remove the LLVM backend so much as make it non-default. It wouldn't solve the issues of the binary size of the zig compiler or difficulties with building it, of course.