.NET Core Linux package feeds (RPM and Debian packages) have expected outages when any package is pushed to them

[dagood]

The Linux package repositories service has a temporary outage whenever packages are pushed to the feed. It depends on how fast syncing can occur, which includes signing the repo data.

• Debian packages: up to 30 minutes is expected

  • Example error message: E: Failed to fetch https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/Packages.bz2 File has unexpected size (72896 != 72828). Mirror sync in progress? [IP: ...]

• RPM packages: up to 2 hours is expected

  • Example error message: Signature verification failed for file 'repomd.xml' from repository 'packages-microsoft-com-prod'.

If you hit this issue and it doesn't resolve itself after the expected time has passed, please file an issue in this repo with the error message and the distro/version you're using.

There unfortunately isn't any status page that gets updated when outages occur.

---

We are tracking adding this known issue to the docs for more visibility: dotnet/docs#16670.

We are trying to work with the Linux package repository team to improve quality of the service. We (.NET Core) and other Microsoft teams depend on this common infrastructure to make our Linux packages available.

I'm planning to leave this issue open to track progress working with the Linux package repository team, and close instances of the problem as a duplicate of this issue.

/cc @dleeapho @leecow @NikolaMilosavljevic @MichaelSimons

[CyrusNajmabadi]

@dagood Do you know what it is in particular about .net core that triggers thsi with these package managers? I've never seen this with a single other package.

We are trying to work with the Linux package repository team to improve quality of the service.

is there an ongoing thread somewhere you can point us to to the communication you're having?

[dagood]

It's all about https://packages.microsoft.com/. (VSCode and PowerShell use it too, for example.) We don't understand and don't yet have transparency into how the service works and why.

The Microsoft linux package repository team has no public presence, so unfortunately there's nothing I can point to. We're going to post info here when we have something to share.

[CyrusNajmabadi]

We don't understand and don't yet have transparency into how the service works and why.

It's very frustrating given the downstream effects that can last hours. Note that this is not a new phenomenon.

[SidShetye]

The Microsoft linux packaging team should look into other packages distributed via apt because I’ve never seen any other package behave this way. Not even the more actively released packages.