Skip to content

default download backend gets repeated connection resets; curl backend does not #5176

Open

Description

@iliana

Verification

Problem

I cannot install Rust toolchains on my machine unless RUSTUP_USE_CURL=1 is set due to connection resets that occur often enough that the retry count is exhausted.

The toolchain version doesn't matter, it just happens to not be installed already and I need to install it.

$ RUSTUP_USE_CURL=0 rustup install 1.89.0
info: syncing channel updates for 1.89.0-x86_64-unknown-linux-gnu
info: latest update on 2025-08-07 for version 1.89.0 (29483883e 2025-08-04)
info: downloading 6 components
        cargo installed                        9.65 MiB
       clippy installed                        4.29 MiB
    rust-docs installed                       20.20 MiB
     rust-std installed                       27.56 MiB
        rustc download failed after 1s
      rustfmt installed                        2.15 MiB
info: rolling back changes
error: component download failed for rustc-x86_64-unknown-linux-gnu: could not download file from 'https://static.rust-lang.org/dist/2025-08-07/rustc-1.89.0-x86_64-unknown-linux-gnu.tar.xz' to '/home/iliana/.rustup/downloads/b42c254e1349df86bd40bc28fdf386172a1a46f2eeabe3c7a08a75cf1fb60e27.partial': error decoding response body: request or response body error: error reading a body from connection: connection reset

(animated recording, shows retries: https://asciinema.org/a/3PnRxeurimNYI6L5)

Using RUSTUP_USE_CURL=1 is reliable. Given that the curl backend is deprecated, I wanted to report the problem I'm having, despite not really understanding what's going on.

Output of RUSTUP_LOG=trace RUSTUP_USE_CURL=0 rustup --quiet install 1.89.0: rustup.log

Steps

  1. RUSTUP_USE_CURL=0 rustup install 1.89.0

Possible Solution(s)

No response

Notes

No response

Rustup version

rustup 1.29.2 (d8c92c676 2026-10-21)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: the currently active `rustc` version is `rustc 1.99.1 (58cf8f9c3 2026-10-13)`

Installed toolchains

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/iliana/.rustup

installed toolchains
--------------------
stable-x86_64-unknown-linux-gnu (active, default)
1.90.0-x86_64-unknown-linux-gnu
1.94.0-x86_64-unknown-linux-gnu
1.94.1-x86_64-unknown-linux-gnu
1.95.0-x86_64-unknown-linux-gnu
1.97-x86_64-unknown-linux-gnu
1.97.1-x86_64-unknown-linux-gnu
1.98.1-x86_64-unknown-linux-gnu
1.99.0-x86_64-unknown-linux-gnu
1.99.1-x86_64-unknown-linux-gnu

active toolchain
----------------
name: stable-x86_64-unknown-linux-gnu
active because: it's the default toolchain
installed targets:
  thumbv6m-none-eabi
  thumbv8m.main-none-eabihf
  x86_64-unknown-linux-gnu

OS version

Gentoo Linux; kernel 6.18.48-gentoo-dist-bin on x86_64

Activity

  1. iliana commented on Oct 29, 2026

    @iliana
    Author

    Some additional investigation:

    • Disabling IPv6 makes the connection resets go away. (I have native IPv6 from my ISP.)
    • Enabling a Tailscale exit node (which reduces the MTU down to 1280) also makes the connection resets go away, even with IPv6 enabled.
  2. rami3l commented on Oct 30, 2026

    @rami3l
    Member

    Looks like this has been reported once at #4837; not sure why this is happening... I'll add this to #3806.

    @djc WDYT?

  3. rami3l commented on Oct 30, 2026

    @rami3l
    Member

    Based on the previous diagnosis in #4837 (comment) I still believe the problem is not reqwest, but http2, because the symptom here should be visible for both libcurl under http2 and our reqwest client. It just so happened that our libcurl backend doesn't have support for http2.

  4. djc commented on Oct 30, 2026

    @djc
    Contributor

    I'm working with Claude to see if it can diagnose the problem and it does have a hypothesis:

    Apparently there's a known issue where, when a path drops traffic briefly, the client's retransmission fires on one of those packets. When that timer fires, Linux picks a new hash for the connection, and on IPv6 that changes the flow label in every packet after it. If some router between the the user and Fastly balances traffic by hashing on the flow label, the retransmitted packet goes to a different Fastly server which doesn't know the connection and sends a reset. (Fastly said in 2018 that they don't include the flow label in hashing itself.)

    With IPv4, there is no flow label, the Tailscale tunnel hides the inner label from routers on the path, and curl is used with HTTP/2 support disabled. Could confirm this by trying with sudo sysctl -w net.ipv6.auto_flowlabels=0 (and the reqwest backend enabled).

    We could disable HTTP/2 with request, but that doesn't seem attractive. We enabled ownload resumption in

    but that doesn't include that particular errors we get from TCP resets during body streaming. Working on a fix for that right now.

  5. iliana commented on Oct 30, 2026

    @iliana
    Author

    sudo sysctl -w net.ipv6.auto_flowlabels=0 makes things reliable with the reqwest backend so that seems to be a reasonable root cause.

    I will also undo that and test the #5177 build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions