Description
While trying to reproduce #1130 yet again, I ran into this failure:
ATTEMPT 12177
running 1 test
test integration_tests::disks::test_disk_create_disk_that_already_exists_fails ... FAILED
failures:
---- integration_tests::disks::test_disk_create_disk_that_already_exists_fails stdout ----
log file: "/dangerzone/omicron_tmp/try_repro.5479/test_all-d586ea57740e3382-test_disk_create_disk_that_already_exists_fails.25655.0.log"
note: configured to log to "/dangerzone/omicron_tmp/try_repro.5479/test_all-d586ea57740e3382-test_disk_create_disk_that_already_exists_fails.25655.0.log"
thread 'integration_tests::disks::test_disk_create_disk_that_already_exists_fails' panicked at 'called `Result::unwrap()` on an `Err` value: BadVersion { expected: "v21.2.9", found: Err(error Some(2) when checking CockroachDB version) }', /home/dap/omicron/test-utils/src/dev/mod.rs:141:42
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The code that produces that message is here:
https://github.com/oxidecomputer/omicron/blob/main/test-utils/src/dev/db.rs#L565-L579
From the code, I think this means that we successfully forked and exec'd cockroach
and the process exited normally (in the Unix sense) with status 2 (which is obviously not normal for what we're doing). I found at least one other issue where cockroach
did this and it looks like Go must have exited with status 2 after catching SIGILL. Unfortunately, the code that runs this check eats both stdout and stderr. A related issue is that if this process had been terminated by a signal, we wouldn't know which one.
So this issue covers updating this code to record the stdout, stderr, and any signal that may have terminated the process.