rustc 1.55.0 intermittently crashes with SIGABRT while compiling the bitfield crate on macOs Monterey

[akscram]

Describe the bug

cargo installed from nixpkgs intermittently fails to build the bitfield crate on macOs Monterey 12.0.1:

ash@cube ~/git/rust-bitfield % cargo build -vv
   Compiling bitfield v0.13.2 (/Users/ash/git/rust-bitfield)
     Running `CARGO=/nix/store/0fgrg47m37b53s5c1bl5lijakcclhcmw-cargo-1.55.0/bin/.cargo-wrapped CARGO_CRATE_NAME=bitfield CARGO_MANIFEST_DIR=/Users/ash/git/rust-bitfield CARGO_PKG_AUTHORS='Loïc Damien <loic.damien@dzamlo.ch>' CARGO_PKG_DESCRIPTION='This crate provides macros to generate bitfield-like struct.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=bitfield CARGO_PKG_REPOSITORY='https://github.com/dzamlo/rust-bitfield' CARGO_PKG_VERSION=0.13.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=13 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/ash/git/rust-bitfield/target/debug/deps:/nix/store/smig7zv3bxqdvgbjd3024gncwkd5923y-rustc-1.55.0/lib:/Users/ash/lib:/usr/local/lib:/usr/lib' rustc --crate-name bitfield src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=c7b624d34760ee7b -C extra-filename=-c7b624d34760ee7b --out-dir /Users/ash/git/rust-bitfield/target/debug/deps -C incremental=/Users/ash/git/rust-bitfield/target/debug/incremental -L dependency=/Users/ash/git/rust-bitfield/target/debug/deps`
error: could not compile `bitfield`

Caused by:
  process didn't exit successfully: `CARGO=/nix/store/0fgrg47m37b53s5c1bl5lijakcclhcmw-cargo-1.55.0/bin/.cargo-wrapped CARGO_CRATE_NAME=bitfield CARGO_MANIFEST_DIR=/Users/ash/git/rust-bitfield CARGO_PKG_AUTHORS='Loïc Damien <loic.damien@dzamlo.ch>' CARGO_PKG_DESCRIPTION='This crate provides macros to generate bitfield-like struct.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=bitfield CARGO_PKG_REPOSITORY='https://github.com/dzamlo/rust-bitfield' CARGO_PKG_VERSION=0.13.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=13 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/ash/git/rust-bitfield/target/debug/deps:/nix/store/smig7zv3bxqdvgbjd3024gncwkd5923y-rustc-1.55.0/lib:/Users/ash/lib:/usr/local/lib:/usr/lib' rustc --crate-name bitfield src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=c7b624d34760ee7b -C extra-filename=-c7b624d34760ee7b --out-dir /Users/ash/git/rust-bitfield/target/debug/deps -C incremental=/Users/ash/git/rust-bitfield/target/debug/incremental -L dependency=/Users/ash/git/rust-bitfield/target/debug/deps` (signal: 6, SIGABRT: process abort signal)

Execution of the rustc command intermittently fails with the following error:

ash@cube ~/git/rust-bitfield % rustc --crate-name bitfield src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=c7b624d34760ee7b -C extra-filename=-c7b624d34760ee7b --out-dir /Users/ash/git/rust-bitfield/target/debug/deps -C incremental=/Users/ash/git/rust-bitfield/target/debug/incremental -L dependency=/Users/ash/git/rust-bitfield/target/debug/deps
rustc(1325,0x700005789000) malloc: *** error for object 0x600002b4ce00: pointer being realloc'd was not allocated
rustc(1325,0x700005789000) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      rustc --crate-name bitfield src/lib.rs --error-format=json  --crate-type lib

Sometimes rustc succeeds but there are very high odds that it fails (no precise measurements but it can fail 4 times from 5 runs). That is, nix-env --upgrade '*' or home-manager switch can very easily fail because bitfield is used as a dependency in many crates.

Steps To Reproduce

There are multiple ways how this problem can be reproduced, e.g. by upgrading nixpkgs or even installing cargo, rustc and rustup nixpkgs for the first time. I find the following one to be more isolated and easily reproducible assuming that cargo and rustc are already installed from nixpkgs:

1. Clone bitfield and make cd there:

git clone git@github.com:dzamlo/rust-bitfield.git
cd rust-bitfield

2. Repeat multiple times clean and build:

cargo clean
cargo build -vv

Expected behavior

bitfield builds successfully and rustc doesn't fail intermittently.

Additional context

This bug was initially reported in rust-lang/cargo/issues/10022 but @YorikSar correctly pointed out that rustc installed as a part of the 1.55.0 toolchain using the official rustup tool doesn't fail:

ash@cube ~/git/rust-bitfield % PATH=~/.rustup/toolchains/1.55.0-x86_64-apple-darwin/bin:$PATH rustc --version --verbose
rustc 1.55.0 (c8dfcfe04 2021-09-06)
binary: rustc
commit-hash: c8dfcfe046a7680554bf4eb612bad840e7631c4b
commit-date: 2021-09-06
host: x86_64-apple-darwin
release: 1.55.0
LLVM version: 12.0.1

ash@cube ~/git/rust-bitfield % PATH=~/.rustup/toolchains/1.55.0-x86_64-apple-darwin/bin:$PATH cargo version --verbose
cargo 1.55.0 (32da73ab1 2021-08-23)
release: 1.55.0
commit-hash: 32da73ab19417aa89686e1d85c1440b72fdf877d
commit-date: 2021-08-23

ash@cube ~/git/rust-bitfield % PATH=~/.rustup/toolchains/1.55.0-x86_64-apple-darwin/bin:$PATH cargo build -vv
   Compiling bitfield v0.13.2 (/Users/ash/git/rust-bitfield)
     Running `CARGO=/Users/ash/.rustup/toolchains/1.55.0-x86_64-apple-darwin/bin/cargo CARGO_CRATE_NAME=bitfield CARGO_MANIFEST_DIR=/Users/ash/git/rust-bitfield CARGO_PKG_AUTHORS='Loïc Damien <loic.damien@dzamlo.ch>' CARGO_PKG_DESCRIPTION='This crate provides macros to generate bitfield-like struct.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=bitfield CARGO_PKG_REPOSITORY='https://github.com/dzamlo/rust-bitfield' CARGO_PKG_VERSION=0.13.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=13 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/ash/git/rust-bitfield/target/debug/deps:/Users/ash/.rustup/toolchains/1.55.0-x86_64-apple-darwin/lib:/Users/ash/lib:/usr/local/lib:/usr/lib' rustc --crate-name bitfield src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=5a03c6b6d187f2dd -C extra-filename=-5a03c6b6d187f2dd --out-dir /Users/ash/git/rust-bitfield/target/debug/deps -C incremental=/Users/ash/git/rust-bitfield/target/debug/incremental -L dependency=/Users/ash/git/rust-bitfield/target/debug/deps`
    Finished dev [unoptimized + debuginfo] target(s) in 0.30s

@afh noted that default user limits for file descriptors in macOs 12 is lesser than in the previous version (256 vs 4864) but raising it to match the previous release didn't help to avoid crashes.

Notify maintainers

@madjar @cstrahan @globin @Havvy

Metadata

ash@cube ~/git/rust-bitfield % nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 21.1.0, macOS 10.16`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.4pre-rc1`
 - channels(ash): `"home-manager, nixpkgs-21.11pre327990.4789953e5c1"`
 - nixpkgs: `/Users/ash/.nix-defexpr/channels/nixpkgs`

ash@cube ~/git/rust-bitfield % rustc --version --verbose
rustc 1.55.0
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-apple-darwin
release: 1.55.0
LLVM version: 12.0.1

ash@cube ~/git/rust-bitfield % cargo --version --verbose
cargo 1.55.0
release: 1.55.0

Maintainer information:

module:
 - cargo
 - rustc
 - rustup

[K900]

This seems to be an OS bug, at least it's gone for me in 12.1 beta.

[YorikSar]

@K900 Could you please check launchctl limit maxfiles on 12.1 beta? Did they change the default again? For me nix-build -A vimPlugins.LanguageClient-neovim '<nixpkgs>' keeps failing at different points with the same SIGABT message.

[siraben]

Also happens with maplit #144692 (comment) and bitvec #144685 (comment) on Monterey (12.0.1) for me.

[siraben]

Some other Rust packages that crash with SIGABRT: nix-template, clap. #144197 (review)

[K900]

I'm getting a 256 soft limit and unlimited hard limit. This should be fine, unless rustc doesn't raise its soft limit?

[afh]

FWIW: Just checked the system user defaults on macOS 11.6.1 and 12.0.1 and both report:

% launchctl limit maxfiles
        maxfiles    256            unlimited

[lilyball]

Has anyone tried catching the signal to see what’s raising it?

[akscram]

Hi @lilyball! SIGABRT is raised by the rustc process itself (see details in the first comment):

ash@cube ~/git/rust-bitfield % rustc --crate-name bitfield src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=c7b624d34760ee7b -C extra-filename=-c7b624d34760ee7b --out-dir /Users/ash/git/rust-bitfield/target/debug/deps -C incremental=/Users/ash/git/rust-bitfield/target/debug/incremental -L dependency=/Users/ash/git/rust-bitfield/target/debug/deps
rustc(1325,0x700005789000) malloc: *** error for object 0x600002b4ce00: pointer being realloc'd was not allocated
rustc(1325,0x700005789000) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      rustc --crate-name bitfield src/lib.rs --error-format=json  --crate-type lib

[kira-bruneau]

It looks like this is the same problem compiling quote & bitvec for texlab on Darwin: #140369.

@174BD7E8 Noticed this bug a while before I could reproduce it on my machine.

[akscram]

Hi @kira-bruneau! It seems that the problem reproduces intermittently rather than constantly and also it seems that some crates have higher chances of reproducing this problem than others. I had to run nix commands under a while loop to get them succeeded, it takes some time though.