sled-agent: move instance configuration generation to Nexus #8002
Conversation
|
This will need a fresh commit hash/SHA from the Propolis repo after oxidecomputer/propolis#899 merges, but I think it is otherwise more or less ready for review (though it could probably use some unit tests of the new virtual platform logic...). |
| //! sled-agent can use to initialize a Propolis VM that exposes the necessary | ||
| //! components. | ||
| //! | ||
| //! For more background on virtual platforms, see RFD 505. |
There was a problem hiding this comment.
considered nitpicky: maybe there ought to be a link to the RFD here?
| //! Each component in a VM spec has a name, which is also its key in the | ||
| //! component map. Generally speaking, Propolis does not care how its clients | ||
| //! name components, but it does require that callers identify those components | ||
| //! by name in API calls that refer to a specific component. To make this as | ||
| //! easy as possible for the rest of Nexus, this module names components as | ||
| //! follows: | ||
| //! | ||
| //! - If a component corresponds to a specific control plane object (i.e. | ||
| //! something like a disk or a NIC that has its own database record and a | ||
| //! corresponding unique identifier): | ||
| //! - If the component requires both a device and a backend, the *backend* | ||
| //! uses the object's ID as a name, and the device uses a module-generated | ||
| //! name. | ||
| //! - If the component is unitary (i.e. it only has one component entry in the | ||
| //! instance spec), this module uses the object ID as its name. | ||
| //! - "Default" components that don't correspond to control plane objects, such | ||
| //! as a VM's serial ports, are named using the constants in the | ||
| //! [`component_names`] module. |
There was a problem hiding this comment.
thank you for documenting these rules, this is lovely.
| /// storage and expect that those devices will always appear in the same places | ||
| /// when the system is stopped and restarted. Changing these mappings for | ||
| /// existing instances may break them! | ||
| fn get_pci_bdf( |
There was a problem hiding this comment.
pretty nitpicky, feel free to disregard me: i would probably drop the "get" from this, i think of "get" as specifically being an accessor/lookup type thing, and the zero_padded_nvme_serial_from_str function in this module doesn't have a get_ prefix...
| backend_id: SpecKey::Uuid(disk.id()), | ||
| pci_path, | ||
| serial_number: zero_padded_nvme_serial_from_str( | ||
| &disk.name().to_string(), |
There was a problem hiding this comment.
couldn't this just be
| &disk.name().to_string(), | |
| &disk.name().as_str(), |
instead of to_stringing it just to throw it away?
| } | ||
|
|
||
| /// A list of named components to add to an instance's spec. | ||
| struct Components(HashMap<String, ComponentV0>); |
There was a problem hiding this comment.
i notice that we use HashMap here but BTreeMap for DisksById, any rationale for that?
|
|
||
| /// Adds the set of disks in the supplied disk list to this component | ||
| /// list. | ||
| fn add_disks(&mut self, disks: DisksById) -> Result<(), Error> { |
There was a problem hiding this comment.
take it or leave it: perhaps we ought to stick a
| fn add_disks(&mut self, disks: DisksById) -> Result<(), Error> { | |
| fn add_disks(&mut self, disks: DisksById) -> Result<(), Error> { | |
| self.0.reserve(disks.0.len() * 2); |
since we know how many entries we're going to add up front?
|
|
||
| /// Adds the supplied set of NICs to this component manifest. | ||
| fn add_nics(&mut self, nics: &[NetworkInterface]) -> Result<(), Error> { | ||
| for nic in nics.iter() { |
There was a problem hiding this comment.
similarly, we could
| for nic in nics.iter() { | |
| self.0.reserve(nics.len() * 2); | |
| for nic in nics.iter() { |
|
|
||
| let mut components = Components::default(); | ||
|
|
||
| let mut volumes = vec![]; |
There was a problem hiding this comment.
also nitpicky: we could save a few potential reallocations by doing
| let mut volumes = vec![]; | |
| let mut volumes = Vec::with_capacity(disks.len()); |
| async fn send_propolis_instance_ensure( | ||
| &self, | ||
| client: &PropolisClient, | ||
| running_zone: &RunningZone, | ||
| migrate: Option<InstanceMigrationTargetParams>, | ||
| ) -> Result<(), Error> { | ||
| // A bit of history helps to explain the workings of the rest of this |
One of the determinations in RFD 505 is that Nexus should be the component that's in charge of determining how to configure a VM given a set of database records describing its instance (the Instance itself, its attached Disks and NetworkInterfaces, etc.). To summarize the rationale in the RFD, the hope is that this will promote two nice properties:
To achieve this:
The main pain point in this change is that sled-agent's API now includes types that it picked up from the propolis-client API, which caused sled-agent's OpenAPI document to balloon with "duplicate" schema descriptions it inherited from propolis-client's generated types. I'm not sure if there's a great way around this (aside from changing the generated Propolis client to
replaceall its generated types with their "native" counterparts); I'm open to suggestions here.Tested by booting a VM in a dev cluster, booting a comparable VM on rack2, and comparing their instance specs (as returned by Propolis's
/instance/specAPI) to make sure they specified the same components with the same configuration.