Skip to content

Make disk location required in sled-agent inventory and Nexus - #11628

Draft
smklein wants to merge 2 commits into
topo-disk-scanfrom
required-disk-location
Draft

smklein wants to merge 2 commits into
topo-disk-scanfrom
required-disk-location

Conversation

@smklein

@smklein smklein commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #11590. Groundwork for #11625.

Since #11590, sled-agent finds disks through the hardware topology, so every disk it reports has a chassis location. This makes that location required beyond sled-agent.

  • Sled-agent API: a new version, MAKE_DISK_LOCATION_REQUIRED, makes InventoryDisk::location a String. Clients of ADD_DISK_LOCATION_TO_INVENTORY still get Some(location).
  • Simulated sled-agent: disks are labelled the way a real sled labels them: N0, N1, … for U.2s, and M.2 East, M.2 West for M.2s. Past the real counts it falls back to N10 or M.2 2. pcie_slot numbering is unchanged, because it's still part of the inventory primary key.
  • Nexus: PhysicalDisk::location is required. Reconfigurator state files without a location no longer load.
  • Database: inv_physical_disk.location stays nullable for now, because rows written by Nexus versions that predate the column have no location. Every new row gets one, and reading a collection skips disks without one. A later release will delete those rows and make the column NOT NULL, ahead of the primary-key change in pcie_slot should not be used as a DB primary key #11625.

Testing: a new simulated sled-agent test covers disk labels, and a new datastore test covers reading a collection that has a NULL location. Existing nexus-types, nexus-inventory, nexus-db-queries (inventory and physical disk), reconfigurator, rack-setup and omdb tests pass.

Sled-agent finds disks through the hardware topology, so every disk it
reports has a chassis location. Add sled-agent API version
MAKE_DISK_LOCATION_REQUIRED, where InventoryDisk::location is a String,
and make nexus_types::inventory::PhysicalDisk::location required.

The simulated sled-agent now labels disks the way a real sled does
(N0.., "M.2 East", "M.2 West"), falling back to N10 or "M.2 2"
past the real bay counts.

inv_physical_disk.location stays nullable for now: rows written by
older Nexus versions have no location. Reading a collection skips
those disks; a later release will delete them and make the column
NOT NULL.
@smklein

smklein commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment on lines +3485 to +3491
// Rows written by Nexus versions that predate the
// `location` column are NULL. These disks are left out
// of the collection.
let Some(location) = disk.location else {
nskipped += 1;
continue;
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Right after an upgrade, omdb nexus sleds expunge-disk can skip its "disk is PRESENT" safety prompt for a disk that is still installed. Reading a collection now silently drops every disk row whose location is NULL. Every collection written before schema version 313 (inv-physical-disk-location) has NULL there. Fix: keep NULL-location disks visible to readers that only check presence, or have the omdb expunge check fall back to inv_physical_disk rows. Another option is to warn when the latest collection skipped disks.

Why this was flagged

The trigger is an operator expunging a disk after the schema migration and before new Nexus finishes its first collection. In that window the latest collection was written by pre-313 Nexus, so location is NULL on every inv_physical_disk row. The read at inventory.rs:3488-3491 hits let Some(location) = disk.location else { nskipped += 1; continue; } and returns sleds with empty disks. dev-tools/omdb/src/bin/omdb/nexus.rs:5734-5740 then finds no match in sleds_containing_disk. Its 0 => {} arm skips the "WARNING: physical disk ... is PRESENT" prompt, so an installed disk is expunged without confirmation. On the base branch the row read back with location: None and the prompt fired. The only signal is a Nexus log warn!, which omdb never sees. The window lasts until the first new collection; inventory.period_secs_collect = 600, but one also runs at startup.

Verification: After schema 313, the latest collection can still have NULL location rows. The diff adds let Some(location) = disk.location else { nskipped += 1; continue; }; in inventory_collection_read, so those disks never reach sled_agents[..].disks. In dev-tools/omdb/src/bin/omdb/nexus.rs:5720-5740, sleds_containing_disk.len() is 0, so the "PRESENT" y/N prompt at 5744-5753 never shows. On the base branch, the disk stayed visible.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant