Skip to content

Add sled-agent API to report multirack join service status - #11535

Merged
andrewjstone merged 6 commits into
mainfrom
sa-mr-join-status
Oct 23, 2026
Merged

andrewjstone merged 6 commits into
mainfrom
sa-mr-join-status

Conversation

@andrewjstone

@andrewjstone andrewjstone commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

We need a way to discover information about sleds in unadopted racks
so that we can show them to operators. The easiest way to do this is to
return that information from the multirack join service, which already
has it. This provides a sled-agent API endpoint to do just that.

It adds a separate watch channel for just the status we care about for
the control plane and utilizes that via the MultirackJoinAccessor,
which was formally the MultirackJoinCompleter.

Now, the question is: How do we find the sled-agent running the
multirack join service? That is going to be solved in a separate PR.
The sled-agent running the join service will publish the new rack subnet
to announce along with its own IP to ddmd. DDMD will announce this
information which will make its way to the proto rack ddmd servers.
Nexuses on the proto rack can then poll their local sled-agent to
discover the location of the sled-agent running the multirack join
service and ask it for this info. Nexus can also complete the multirack
join via the same sled-agent when the user adopts the new rack into the
cluster. See oxidecomputer/maghemite#939

We need a way to discover information about sleds in unadopted racks
so that we can show them to operators. The easiest way to do this is to
return that information from the multirack join service, which already
has it. This provides a sled-agent API endpoint to do just that.

It adds a separate watch channel for just the status we care about for
the control plane and utilizes that via the `MultirackJoinAccessor`,
which was formally the `MultirackJoinCompleter`.

Now, the question is: How do we find the sled-agent running the
multirack join service? That is going to be solved in a separate PR.
The sled-agent running the join service will publish the new rack subnet
to announce along with its own IP to ddmd. DDMD will announce this
information which will make its way to the proto rack ddmd servers.
Nexuses on the proto rack can then poll their local sled-agent to
discover the location of the sled-agent running the multirack join
service and ask it for this info. Nexus can also complete the multirack
join via the same sled-agent when the user adopts the new rack into the
cluster. See oxidecomputer/maghemite#939
Comment thread sled-agent/src/bootstrap/rack_ops.rs Resolved
Comment thread sled-agent/multirack-join/src/lib.rs Resolved
Comment thread sled-agent/multirack-join/src/lib.rs Resolved
Comment thread sled-agent/multirack-join/src/lib.rs Resolved
Comment thread sled-agent/src/bootstrap/server.rs Resolved

@jgallagher jgallagher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all the change LGTM. I assume sorting out #11535 (comment) will be minor, so happy to go ahead and approve.

@nickelization nickelization left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment thread sled-agent/api/src/lib.rs

/// Return the status of this sled's multirack join service.
///
/// Returns a 404 if no multirack join service is running, or a 409 if the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I can't remember ever seeing 409 used before, always fun seeing unusual status codes in places that actually make sense for them :)

fn from(value: StartSledAgentsStatus) -> Self {
let sled_id = value.this_nodes_sled_id;
let rack_id = value.rack_id;
let rack_subnet = Ipv6Subnet::<RACK_PREFIX_LENGTH>::new(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Genuinely not sure if this is worth it, but I wonder if it would be a good idea to check that all the sleds have the same prefix and log a warning or something here if they don't?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They all get generated from the same prefix, so they share it by construction. This is just to extract it back out for the API.

While it doesn't matter here, I do think that Nexus should validate this when it receives the response containing this data.

@andrewjstone
andrewjstone enabled auto-merge (squash) October 23, 2026 15:17
@andrewjstone
andrewjstone merged commit d31af1e into main Oct 23, 2026
20 checks passed
@andrewjstone
andrewjstone deleted the sa-mr-join-status branch October 23, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants