`https://apt.llvm.org/llvm.sh 18` no longer installs llvm-18

[colesbury]

Until a few hours ago (July 30, 2024 15:54  UTC) the following invocation installed the llvm-18 package:

wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 18

The missing package means that llvm-symbolizer is missing, which can lead to breakages when using sanitizers. For example, CPython has a thread sanitizer CI that suddenly broke because the suppressions file (and stack traces) no longer work.

For context, the same problem (missing llvm-symbolizer) happened around July 17 and seemed to be resolved around July 22. See also:

• Ubuntu 20.04 llvm.sh 18 all Fails Due To Unresolvable Interdependencies #99502
• Thread sanitizer tests failing in GitHub actions python/cpython#121946

Note that ./llvm.sh 17 seems to continue to install llvm-symbolizer.

On Ubuntu 22.04 (Jammy):

Before

...
+ apt-get install -y clang-18 lldb-18 lld-18 clangd-18
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libabsl20210324 libclang-common-18-dev libclang-cpp18 libclang-rt-18-dev
  libclang1-18 libgrpc++1 libgrpc10 liblldb-18 libllvm18 libprotobuf23 llvm-18
  llvm-18-dev llvm-18-linker-tools llvm-18-runtime llvm-18-tools
  python3-lldb-18
Suggested packages:
  clang-18-doc wasi-libc llvm-18-doc

After:

...
+ apt-get install -y clang-18 lldb-18 lld-18 clangd-18
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libabsl20210324 libclang-common-18-dev libclang-cpp18 libclang-rt-18-dev
  libclang1-18 libgrpc++1 libgrpc10 liblldb-18 libllvm18 libprotobuf23
  llvm-18-linker-tools python3-lldb-18
Suggested packages:
  clang-18-doc wasi-libc
Recommended packages:
  llvm-18-dev

EDIT: llvm-18-dev -> llvm-18. The llvm-18 package is also no longer installed, which seems more relevant.

[0xE1E10]

I don't see anything wrong in llvm.sh. llvm-18 is not installed because it's a dep of llvm-18-dev and llvm-18-dev failed to install:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libllvm18 : Breaks: llvm-18-dev (< 1:18.1.8~++20240730104741) but 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 is to be installed
E: Unable to correct problems, you have held broken packages.

I don't understand why the condition fails to be honest. Compare versions says it's valid:

❯ if dpkg --compare-versions '1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143' '<' '1:18.1.8~++20240730104741'; then echo great; fi
dpkg: warning: --compare-versions used with obsolete relation operator '<'
great

But apt insist it's not:

❯ sudo apt-get -s -o Debug::pkgProblemResolver=yes install llvm-18-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Starting pkgProblemResolver with broken count: 1
Starting 2 pkgProblemResolver with broken count: 1
Investigating (0) libllvm18:amd64 < 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 @ii pmK Ib >
Broken libllvm18:amd64 Breaks on llvm-18-dev:amd64 < none -> 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 @un pumN > (< 1:18.1.8~++20240730104741)
  Considering llvm-18-dev:amd64 10002 as a solution to libllvm18:amd64 35
Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libllvm18 : Breaks: llvm-18-dev (< 1:18.1.8~++20240730104741) but 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 is to be installed
E: Unable to correct problems, you have held broken packages.

[ian-collins-ctct]

It fails in a simple docker test case, Dockerfile:

FROM ubuntu:22.04
LABEL owner=build

# NOTE: Keep the package lines sorted when adding lines.
RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get --yes install \
    lsb-release \
    gnupg \
    software-properties-common \
    wget \
    && rm -rf /var/lib/apt/lists/*

ARG CLANG_VERSION
ENV CLANG_VERSION=$CLANG_VERSION

# Official LLVM repo to get a later version of clang than is available in focal.
RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh \
    && ./llvm.sh "$CLANG_VERSION" all \
    && rm -rf ./llvm.sh /var/lib/apt/lists/*

Error:

The following packages have unmet dependencies:
 libllvm18 : Breaks: llvm-18-dev (< 1:18.1.8~++20240730104741) but 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 is to be installed
E: Unable to correct problems, you have held broken packages.

[PatrickKa]

I ran into the same, or at least a very similar, problem yesterday. My GitHub Actions workflow suddenly broke because installing LLVM 18 on the ubuntu-22.04 runners stopped working due to the same error.

The following packages have unmet dependencies:
   libllvm18 : Breaks: llvm-18-dev (< 1:18.1.8~++20240730104741) but 1:18.1.8~++20240730092223+3b5b5c1ec4a3-1~exp1~20240730092238.143 is to be installed

I kinda hoped someone will fix the problem overnight, but unfortunately that didn't happen 😅. Any help or workarounds?

[ian-collins-ctct]

In my case, I used a local copy and updated the $PKG= line to install the bits I wanted rather than passing "all".

PKG="clang-$LLVM_VERSION clang-tidy-$LLVM_VERSION lld-$LLVM_VERSION llvm-$LLVM_VERSION"

A less than ideal hack to get me unstuck!

[sylvestre]

I will fix that later today.

[PatrickKa]

Thx. In the meantime, I upgraded to ubuntu-24.04 runners which already comes with LLVM 18 preinstalled, so that "solved" the problem for me.