Skip to content

docs: add procedures for disabling loopback workaround #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 1, 2025
Merged

docs: add procedures for disabling loopback workaround #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5fbae19
docs: add procedures for disabling loopback workaround
mrbojangles3 Jul 1, 2025
a97d685
integrate feedback
mrbojangles3 Jul 1, 2025
40596f4
docs: update known issue re loopback workaround
edipascale Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/install-upgrade/upgrade.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,53 @@
# Upgrade

## Upgrades removing loopback connections
Starting with Fabric 25.03 and Broadcom SONiC 4.5.0, loopback connections are no longer
required for local VPC peering. A feature flag has been created to assist in a
smooth removal process, `loopbackWorkaroundDisable`, which takes a boolean
value.

### Fresh installs of 25.03

Users in a greenfield environment installing version 25.03 of the Fabric will
see that `hhfab init` will supply the needed option. For users who are
generating configuration in another way ensure that
`loopbackWorkaroundDisable: true` is inside `.spec.config.fabric` of the
Fabricator object.


### Upgrades to 25.03

For users upgrading to version 25.03 follow the steps:

0. Upgrade to 25.03 using steps from [previous
versions](#upgrades-from-beta-1-onwards)

0. Ensure all switch agents are on v0.80.0 (or greater)
* `kubectl get agents` to see the version

0. Upgrade the switch operating systems to 4.5.0
* the switch reinstall feature of k9s is very useful for this purpose

0. Confirm all the switches are up and running SONiC 4.5.0
* `kubectl get agents -o wide`, look at the Software column

0. Run the command to patch the fabricator object
* `kubectl patch -n fab --type merge fabricator/default -p '{"spec":{"config":{"fabric":{"loopbackWorkaroundDisable":true}}}}'`

0. Ensure that all agents have converged:
* `kubectl get agents -o wide`, ensure that columns `currentg` equals `appliedg`

0. Now the loopback connections can be removed from the logical wiring:
* `kubectl get connections | grep loopback`
* `kubectl delete connections/leaf-03--vpc-loopback`
0. Finally, unplug loopback cables as needed


### Beyond 25.03
Starting in 25.04 the presence of `loopbackWorkaroundDisable: true` will be required
in order for updates to take place.


## Upgrades from Beta-1 onwards

Starting with Beta-1 release and onwards, the upgrade process is more streamlined and fully automated. The control node
Expand Down
8 changes: 6 additions & 2 deletions docs/known-limitations/known-limitations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,15 @@ s5248-01(config-if-Ethernet1)# no shutdown
### VPC local peering can cause the agent to fail if subinterfaces are not supported on the switch

As explained in the [Architecture page](../architecture/fabric.md#vpc-peering), to workaround
limitations in SONiC, a peering between two VPCs (or a VPC and an External) which are both
limitations in older versions of SONiC, a peering between two VPCs (or a VPC and an External) which are both
attached to the peering switch is implemented over a pair of loopback interfaces.
This workaround requires subinterface support on the switch where the peering is being
instantiated. If the affected switch does not meet this requirement, the agent will fail
to apply the desired configuration.

!!! note
Starting from Fabric version 25.03, the loopback workaround is no longer needed.
Comment on lines +67 to +68
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then if it's no longer needed, why mention it at all in the docs and have this limitation listed?? Can't we just keep it in the stable branches of the docs? (We had this discussion for another limitation before.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because people might read the latest docs even when they are on old releases where the workaround was needed and/or enabled by default, and so it's better for them to have this information here too (imho)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an addendum, when we no longer support versions of the fabric where these limitation applied, then I would be totally in favor of removing it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because people might read the latest docs even when they are on old releases where the workaround was needed and/or enabled by default

If they run old releases they should read the docs for the old release 🤷 and the old release should list the workarounds to use. It seems confusing to me to keep limitations in the docs that no longer apply.

when we no longer support versions of the fabric where these limitation applied, then I would be totally in favor of removing it

Yes please, or we'll end up with notes for all limitations removed since the early days of the project 🫠


#### Diagnosing this issue

The applied generation of the affected agent reported by kubectl will not
Expand All @@ -74,7 +77,8 @@ converge to the last desired generation. Additionally, the agent logs on the swi

#### Known workarounds

Configure remote VPCPeering instead of local peering in any instance where both
If possible, [upgrade to 25.03 and disable the loopback workaround](../install-upgrade/upgrade.md#upgrades-to-2503).
Alternatively, configure remote VPCPeering instead of local peering in any instance where both
peering elements are locally attached and the target switch does not support subinterfaces.
You can double-check whether your switch model meets this requirement by looking at the
[Switch Profiles Catalog](../reference/profiles.md) entry for it.
Expand Down