Repository navigation
Conversation
The hardware poll now starts from libtopo's bay and M.2 slot nodes and asks libnvme only for what the topology does not record: the PCI vendor id, the active namespace count, and the firmware slots. The devinfo blkdev walk is gone. devinfo still supplies the sled type, boot-storage-unit, baseboard, and tofino state, none of which overlap what the topology says about disks. A failed snapshot aborts the poll and keeps the previous view, as a libnvme failure already did. So does a snapshot that shows a device the topology could not describe: a bay whose bridge has an nvme driver but no nvme node, an nvme node without an instance, an incomplete disk node, or a controller with active namespaces the topology has no disk for. A controller with no namespace, or a non-NVMe device, is reported as a bay occupant rather than an error. DiskVariant comes from the bay kind (bay or M.2 slot), so the per-board PCIe slot range tables are deleted; the boot slot mapping stays. Every disk now has a location, and HardwareView carries each bay's occupancy as DiskBay for inventory to report.
Sled Agent API version ADD_DISK_BAYS_TO_INVENTORY adds Inventory::disk_bays, one entry per U.2 bay and M.2 socket of the chassis: empty, holding a managed disk, or holding a device that is not a usable disk (an NVMe controller with no namespace, or something that is not NVMe). It also makes InventoryDisk::location required, since the topology that supplies the label is now where sled-agent finds disks. The bays travel from HardwareView through a DiskBaysSender in the config reconciler to the inventory endpoint. The sim sled-agent reports one bay per simulated disk, labelled with the same rule synthetic disks use, and sled-storage exposes that rule as synthetic_disk_location. Nexus-side fixtures gain the new field; the example collection describes all twelve bays of its first sled.
Schema 303 adds inv_disk_bay, one row per U.2 bay or M.2 socket a sled reported, keyed by collection, sled, and chassis location, with an inv_disk_bay_occupant enum saying whether the bay is empty, holds a managed disk (identified like inv_physical_disk), or holds a device that is not a usable disk. The inventory collector converts the sled-agent bays into nexus-types DiskBay values, sorted by location so a collection equals its own database round trip, and the datastore inserts, reads, and prunes them alongside physical disks. The inventory display gains a "disk bays:" section and omdb gains "db inventory disk-bays". The example collection describes all twelve bays of its first sled, including an NVMe controller with no namespace and an unrecognized device, which also keeps the populated-tables check covering the new table. The planning fixture reports one occupied bay per fake disk.
A doc comment quoted the synthetic M.2 label as "M.2 <ordinal>", which rustdoc reads as an unclosed HTML tag, and the disk_bay module doc linked to a private item.
M2Slot::from_chassis_location maps "M.2 East" to A and "M.2 West" to B, replacing the TryFrom<i64> impl that keyed on the PCIe slot numbers of the bridges above the M.2 sockets. Those numbers are internal to the board and may change across host OS versions; the chassis labels are what the hardware topology promises. On Gimlet and Cosmo the two mappings agree, so real sleds see no change. Synthetic M.2s named m2_0 and m2_1 are labelled East and West, so they now get slots A and B where before their offset PCIe slots matched neither.
A bay the topology or libnvme could only partly describe used to fail the whole hardware poll. At runtime that froze every other bay's view; at startup it failed HardwareManager::new, which sled-agent unwraps. Now such a bay is unreadable for that poll only: it keeps what the previous view said about it, or is reported as a device with no disk if there is no previous view. The rest of the poll goes ahead. This covers a disk node missing its blkdev properties (as while sled-agent formats a U.2), an NVMe device the topology did not enumerate, an nvme node without a driver instance, more than one occupant, a libnvme failure, and a disk identity seen in two bays. If the bays cannot be read at all (libtopo or libnvme cannot be opened), the previous view of disks stands and the rest of the poll, including the Tofino, still updates, so this no longer fails startup. Namespaces are now counted at the NotIgnored level. The nvme driver ignores namespaces it cannot attach blkdev to (such as ones formatted with protection information) and the topology describes none of them, so counting at Active reported such a drive as a missing namespace on every poll.
- The inventory builder keeps only the first bay with a given location and records a collection error, since a duplicate would violate the inv_disk_bay primary key and fail the whole collection's insert. - Test M2Slot::from_chassis_location. - Label M.2 test disks with an M.2 location rather than "N0". - Name the API version in nexus-types comments rather than its number.
- HardwareView records whether its latest poll described every disk bay, and installinator treats an incomplete view as a failed scan, so its retries cover a bay that is briefly unreadable rather than installing to only one M.2. - When two bays claim the same disk, one that saw it on this poll wins over one carrying it over from the previous view, since the disk may have moved. - The inventory builder keeps the bay holding a disk when a sled reports a location twice. - Document that a device occupant can also be one sled-agent could not read, and drop the stale 'active namespace' wording. - Log a locked NVMe controller once per poll rather than twice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #11279.
sled-agent now finds disks through the hardware topology rather than devinfo, so every disk has a chassis location. Inventory also reports every U.2 bay and M.2 socket, including empty ones. Fixes #11319, and makes progress on #11258.
HardwareViewreports when its picture of the disks is incomplete, and installinator retries until it isn't.ADD_DISK_BAYS_TO_INVENTORY): AddsInventory::disk_bays. Each entry is an empty bay, a managed disk, or a device that isn't a usable disk.InventoryDisk::locationbecomes required. Older versions are served by converting down.inv-disk-baymigration): Adds aninv_disk_baytable, a "disk bays" section in the inventory display, andomdb db inventory disk-bays. A sled reporting the same location twice records a collection error instead of failing the insert.M2Slot::from_chassis_locationreplaces the PCIe-slot-basedTryFrom<i64>. On Gimlet and Cosmo the two mappings agree. Synthetic M.2s (m2_0/m2_1) now get slots A/B, so on a4x2 host phase 2 reports a different (still failing) error.Not included: deriving
is_boot_diskfrom the kernel'sboot-sourceinstead ofbootdisk_pcie_slots(); droppingpcie_slotfrom the inventory primary key; exposing locations in the external API (#11088).Testing: unit tests for the affected crates (including the per-bay resolution and builder dedupe); the illumos-only code builds on illumos. Not yet run on real hardware.