Skip to content

cockroach version check should include stdout/stderr and signal information on failure #1279

Closed
@davepacheco

Description

@davepacheco

While trying to reproduce #1130 yet again, I ran into this failure:

ATTEMPT 12177

running 1 test
test integration_tests::disks::test_disk_create_disk_that_already_exists_fails ... FAILED

failures:

---- integration_tests::disks::test_disk_create_disk_that_already_exists_fails stdout ----
log file: "/dangerzone/omicron_tmp/try_repro.5479/test_all-d586ea57740e3382-test_disk_create_disk_that_already_exists_fails.25655.0.log"
note: configured to log to "/dangerzone/omicron_tmp/try_repro.5479/test_all-d586ea57740e3382-test_disk_create_disk_that_already_exists_fails.25655.0.log"
thread 'integration_tests::disks::test_disk_create_disk_that_already_exists_fails' panicked at 'called `Result::unwrap()` on an `Err` value: BadVersion { expected: "v21.2.9", found: Err(error Some(2) when checking CockroachDB version) }', /home/dap/omicron/test-utils/src/dev/mod.rs:141:42
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The code that produces that message is here:
https://github.com/oxidecomputer/omicron/blob/main/test-utils/src/dev/db.rs#L565-L579

From the code, I think this means that we successfully forked and exec'd cockroach and the process exited normally (in the Unix sense) with status 2 (which is obviously not normal for what we're doing). I found at least one other issue where cockroach did this and it looks like Go must have exited with status 2 after catching SIGILL. Unfortunately, the code that runs this check eats both stdout and stderr. A related issue is that if this process had been terminated by a signal, we wouldn't know which one.

So this issue covers updating this code to record the stdout, stderr, and any signal that may have terminated the process.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions