ICE in laby 0.2.4: we should not pop and push a frame in one step

[saethlin]

I have spent hours trying to reproduce this outside of my crater-but-miri project: https://github.com/saethlin/miri-tools and I have had no success yet. Maybe I'll try to reduce this further into something that's easy to reproduce.

What I have learned so far is that it appears that I need:

export TERM=xterm-256color
export MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-retag-fields"
cargo miri test -- --test-threads=2

But again, I swear I've manually spun up the same setup in the Docker containers that project spins up and yet, no ICE when doing things manually.

thread 'rustc' panicked at 'assertion failed: `(left == right)`
  left: `Instance { def: Item(WithOptConstParam { did: DefId(23:1289 ~ test[1687]::__rust_begin_short_backtrace), const_param_did: None }), substs: [fn()] }`,
 right: `Instance { def: Item(WithOptConstParam { did: DefId(5:452 ~ alloc[c80e]::boxed::{impl#0}::new), const_param_did: None }), substs: [alloc::sync::ArcInner<std::sync::Mutex<std::option::Option<[closure@test::run_test::run_test_inner::{closure#0}]>>>] }`: we should not pop and push a frame in one step', src/diagnostics.rs:440:17
stack backtrace:
   0:     0x7f1b252a4210 - std::backtrace_rs::backtrace::libunwind::trace::hf2278ed4f868ae96
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   1:     0x7f1b252a4210 - std::backtrace_rs::backtrace::trace_unsynchronized::h7f40d5d813898dd4
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f1b252a4210 - std::sys_common::backtrace::_print_fmt::he51a58187d9ae345
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys_common/backtrace.rs:66:5
   3:     0x7f1b252a4210 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc389f980b6cb07fb
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys_common/backtrace.rs:45:22
   4:     0x7f1b252ff02e - core::fmt::write::h78d46ef62b9f9968
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/core/src/fmt/mod.rs:1202:17
   5:     0x7f1b25294ad5 - std::io::Write::write_fmt::h6ee421ee7580e814
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/io/mod.rs:1679:15
   6:     0x7f1b252a6e63 - std::sys_common::backtrace::_print::h8952290f5048174d
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys_common/backtrace.rs:48:5
   7:     0x7f1b252a6e63 - std::sys_common::backtrace::print::h9e1e4f27fa020a6f
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys_common/backtrace.rs:35:9
   8:     0x7f1b252a6e63 - std::panicking::default_hook::{{closure}}::h1688ea5adf02a058
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/panicking.rs:295:22
   9:     0x7f1b252a6b4f - std::panicking::default_hook::h767527990525c9aa
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/panicking.rs:314:9
  10:     0x7f1b27b17511 - rustc_driver[7ef72a67e6ebc4c7]::DEFAULT_HOOK::{closure#0}::{closure#0}
  11:     0x7f1b252a769d - std::panicking::rust_panic_with_hook::h9c7dee39dc16e51e
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/panicking.rs:702:17
  12:     0x7f1b252a74f7 - std::panicking::begin_panic_handler::{{closure}}::haffc5420dfe12689
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/panicking.rs:588:13
  13:     0x7f1b252a46ec - std::sys_common::backtrace::__rust_end_short_backtrace::h361b157e6d2c66f3
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys_common/backtrace.rs:138:18
  14:     0x7f1b252a7212 - rust_begin_unwind
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/panicking.rs:584:5
  15:     0x7f1b252fbc03 - core::panicking::panic_fmt::hb02dec70af640359
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/core/src/panicking.rs:142:14
  16:     0x7f1b252fbef1 - core::panicking::assert_failed_inner::ha7d8c78d8bbfdf2d
  17:     0x5632e1d3462b - core::panicking::assert_failed::hf6d3d60680fa8652
  18:     0x5632e1ece2c4 - std::thread::local::LocalKey<T>::with::h76222bfafb64843f
  19:     0x5632e1df2faa - std::panic::catch_unwind::ha726501a75eb6794
  20:     0x5632e1dedd5e - miri::eval::eval_entry::h185ca269a1c49073
  21:     0x5632e1d4042f - rustc_interface::passes::QueryContext::enter::h8895c980004f76f2
  22:     0x5632e1d4607f - <miri::MiriCompilerCalls as rustc_driver::Callbacks>::after_analysis::ha3b259c44677c415
  23:     0x7f1b268de327 - rustc_interface[7b82b5b949e58a29]::interface::create_compiler_and_run::<core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>, rustc_driver[7ef72a67e6ebc4c7]::run_compiler::{closure#1}>
  24:     0x7f1b268dcc51 - <scoped_tls[ae248541ea4499d]::ScopedKey<rustc_span[ee7279913b705f64]::SessionGlobals>>::set::<rustc_interface[7b82b5b949e58a29]::interface::run_compiler<core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>, rustc_driver[7ef72a67e6ebc4c7]::run_compiler::{closure#1}>::{closure#0}, core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>>
  25:     0x7f1b268dc93f - std[c3d32730625157ed]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[7b82b5b949e58a29]::util::run_in_thread_pool_with_globals<rustc_interface[7b82b5b949e58a29]::interface::run_compiler<core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>, rustc_driver[7ef72a67e6ebc4c7]::run_compiler::{closure#1}>::{closure#0}, core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>>::{closure#0}, core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>>
  26:     0x7f1b27988659 - <<std[c3d32730625157ed]::thread::Builder>::spawn_unchecked_<rustc_interface[7b82b5b949e58a29]::util::run_in_thread_pool_with_globals<rustc_interface[7b82b5b949e58a29]::interface::run_compiler<core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>, rustc_driver[7ef72a67e6ebc4c7]::run_compiler::{closure#1}>::{closure#0}, core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>>::{closure#0}, core[248183d12c27127]::result::Result<(), rustc_errors[c5112f0e4f8e3032]::ErrorGuaranteed>>::{closure#1} as core[248183d12c27127]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  27:     0x7f1b252b14f3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h61538f7aea70034d
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/alloc/src/boxed.rs:1940:9
  28:     0x7f1b252b14f3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hf97ae58abf16e3b8
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/alloc/src/boxed.rs:1940:9
  29:     0x7f1b252b14f3 - std::sys::unix::thread::Thread::new::thread_start::h54cb802344e1fe9c
                               at /rustc/e7c7aa7288559f8e5ea7ce3543ff946b09783628/library/std/src/sys/unix/thread.rs:108:17
  30:     0x7f1b25152609 - start_thread
  31:     0x7f1b24f26163 - clone
  32:                0x0 - <unknown>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.65.0-nightly (e7c7aa728 2022-09-07) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C linker=clang -C incremental -C link-arg=-fuse-ld=lld -Z miri-disable-isolation -Z miri-retag-fields

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

Miri caused an ICE during evaluation. Here's the interpreter backtrace at the time of the panic:
note: the place in the program where the ICE was triggered
  --> /root/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/laby_common-0.1.1/src/internal/buffer.rs:79:45
   |
79 |             data: align_of::<u8>() as *mut u8, // dangling pointer
   |                                             ^
   |
   = note: inside `laby::internal::Buffer::new` at /root/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/laby_common-0.1.1/src/internal/buffer.rs:79:45
   = note: inside `laby::internal::escape_str` at /root/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/laby_common-0.1.1/src/internal/escape.rs:160:22
note: inside `all` at tests/escape.rs:13:13
  --> tests/escape.rs:13:13
   |
13 |     let s = escape_str("&\"'<>");
   |             ^^^^^^^^^^^^^^^^^^^^
note: inside closure at tests/escape.rs:12:1
  --> tests/escape.rs:12:1
   |
11 |   #[test]
   |   ------- in this procedural macro expansion
12 | / fn all() {
13 | |     let s = escape_str("&\"'<>");
14 | |     assert_eq!(s, "&amp;&quot;&#39;&lt;&gt;");
15 | | }
   | |_^
   = note: inside `<[closure@tests/escape.rs:12:1: 15:2] as std::ops::FnOnce<()>>::call_once - shim` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:572:5
   = note: inside closure at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:563:30
   = note: inside `<[closure@test::run_test::{closure#1}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1940:9
   = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
   = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
   = note: inside `test::run_test_in_process` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:595:18
   = note: inside closure at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:489:39
   = note: inside closure at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:516:37
   = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<[closure@test::run_test::run_test_inner::{closure#1}], ()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:122:18
   = note: inside closure at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/thread/mod.rs:514:17
   = note: inside `<std::panic::AssertUnwindSafe<[closure@std::thread::Builder::spawn_unchecked_<[closure@test::run_test::run_test_inner::{closure#1}], ()>::{closure#1}::{closure#0}]> as std::ops::FnOnce<()>>::call_once` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<[closure@std::thread::Builder::spawn_unchecked_<[closure@test::run_test::run_test_inner::{closure#1}], ()>::{closure#1}::{closure#0}]>, ()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
   = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<[closure@std::thread::Builder::spawn_unchecked_<[closure@test::run_test::run_test_inner::{closure#1}], ()>::{closure#1}::{closure#0}]>>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<[closure@std::thread::Builder::spawn_unchecked_<[closure@test::run_test::run_test_inner::{closure#1}], ()>::{closure#1}::{closure#0}]>, ()>` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
   = note: inside closure at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/thread/mod.rs:513:30
   = note: inside `<[closure@std::thread::Builder::spawn_unchecked_<[closure@test::run_test::run_test_inner::{closure#1}], ()>::{closure#1}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `<std::boxed::Box<dyn std::ops::FnOnce()> as std::ops::FnOnce<()>>::call_once` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1940:9
   = note: inside `<std::boxed::Box<std::boxed::Box<dyn std::ops::FnOnce()>> as std::ops::FnOnce<()>>::call_once` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1940:9
   = note: inside `std::sys::unix::thread::Thread::new::thread_start` at /root/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:108:17
   = note: this note originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

error: test failed, to rerun pass `--test escape`

[RalfJung]

You said this needs --test-threads=2? Maybe the non-halting diagnostic infrastructure is actually subtly incompatible with having multiple threads... it predates support for concurrency and some of the assumptions it makes about reconstructing the stack trace from the time the diagnostic was triggered might just be wrong nowadays.