RTX 3090 rear fan intermittently runs at 100% (noisy) although NVML target is 30%

[Sylva1n]

RTX 3090 rear fan intermittently runs at 100% although NVML target is 30%

I am looking for help from people who know RTX 3090 fan behavior and fan-control diagnostics well.

Context

I have a machine with three NVIDIA GeForce RTX 3090 GPUs.

The issue concerns GPU 2, identified as:

index, uuid, pci.bus_id, name
0, GPU-672ea43d-ba25-9650-b494-9787bd455159, 00000000:02:00.0, NVIDIA GeForce RTX 3090
1, GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf, 00000000:81:00.0, NVIDIA GeForce RTX 3090
2, GPU-5f33c236-dca2-c873-2886-c0e2aa54a727, 00000000:82:00.0, NVIDIA GeForce RTX 3090

For GPU 2:

82:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: NVIDIA Corporation Device 147d
        Physical Slot: 5
                LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
                LnkSta: Speed 2.5GT/s (downgraded), Width x16
                LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
        Kernel driver in use: nvidia
        Kernel modules: nvidia_drm, nvidia

Driver version:

index, uuid, driver_version, name
0, GPU-672ea43d-ba25-9650-b494-9787bd455159, 580.126.20, NVIDIA GeForce RTX 3090
1, GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf, 580.126.20, NVIDIA GeForce RTX 3090
2, GPU-5f33c236-dca2-c873-2886-c0e2aa54a727, 580.126.20, NVIDIA GeForce RTX 3090

Symptom

The behavior is intermittent.

Sometimes the GPU fan behavior appears normal after boot.

On other boots, the rear fan of GPU 2 physically runs at 100%. Then fan is very noisy at that speed.

I have not found a clear trigger yet.

Earlier observed state

At one point, there was no meaningful GPU load, but the rear fan problem appeared.

Command:

nvidia-smi --query-gpu=index,name,temperature.gpu,temperature.memory,pstate,power.draw,fan.speed,clocks.gr,clocks.mem,utilization.gpu --format=csv

Output:

index, name, temperature.gpu, temperature.memory, pstate, power.draw [W], fan.speed [%], clocks.current.graphics [MHz], clocks.current.memory [MHz], utilization.gpu [%]
0, NVIDIA GeForce RTX 3090, 33, N/A, P8, 26.40 W, 0 %, 210 MHz, 405 MHz, 0 %
1, NVIDIA GeForce RTX 3090, 23, N/A, P8, 8.46 W, 0 %, 210 MHz, 405 MHz, 0 %
2, NVIDIA GeForce RTX 3090, 22, N/A, P8, 14.59 W, 0 %, 210 MHz, 405 MHz, 0 %

At the same time, this custom NVML fan feedback tool reported:

GPU  FAN  TARGET  FEEDBACK  DELTA  STATUS
---  ---  ------  --------  -----  --------
  1    0     30%        0%   +30%  mismatch
  1    1     30%        0%   +30%  mismatch
  2    0     30%        0%   +30%  mismatch
  2    1     30%      100%   -70%  mismatch

So for GPU 2 / fan 1, the NVML target was 30%, while feedback was 100%.

Later observed state

On another boot, the driver and nvidia-smi showed GPU 2 at 30% fan speed globally:

printf '%s\n' '--- driver ---' '--- nvidia-smi -q errors ---'; nvidia-smi --query-gpu=index,uuid,driver_version,name --format=csv; nvidia-smi -q | rg -i 'GPU UUID|Product Name|Fan Speed|Unknown Error|Error|Power State|Temperature|PCI'

Output:

--- driver ---
--- nvidia-smi -q errors ---
index, uuid, driver_version, name
0, GPU-672ea43d-ba25-9650-b494-9787bd455159, 580.126.20, NVIDIA GeForce RTX 3090
1, GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf, 580.126.20, NVIDIA GeForce RTX 3090
2, GPU-5f33c236-dca2-c873-2886-c0e2aa54a727, 580.126.20, NVIDIA GeForce RTX 3090
    Product Name                                       : NVIDIA GeForce RTX 3090
    GPU UUID                                           : GPU-672ea43d-ba25-9650-b494-9787bd455159
    PCI
            PCIe Generation
    Fan Speed                                          : 0 %
    ECC Errors
            SRAM PCIE                                  : N/A
    Temperature
        GPU Target Temperature                         : 83 C
    Product Name                                       : NVIDIA GeForce RTX 3090
    GPU UUID                                           : GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf
    PCI
            PCIe Generation
    Fan Speed                                          : 30 %
    ECC Errors
            SRAM PCIE                                  : N/A
    Temperature
        GPU Target Temperature                         : 83 C
    Product Name                                       : NVIDIA GeForce RTX 3090
    GPU UUID                                           : GPU-5f33c236-dca2-c873-2886-c0e2aa54a727
    PCI
            PCIe Generation
    Fan Speed                                          : 30 %
    ECC Errors
            SRAM PCIE                                  : N/A
    Temperature
        GPU Target Temperature                         : 83 C

Then the custom NVML fan feedback tool reported:

GPU  FAN  TARGET  FEEDBACK  DELTA  STATUS
---  ---  ------  --------  -----  --------
  1    0     30%       30%    +0%  ok
  1    1     30%       30%    +0%  ok
  2    0     30%       30%    +0%  ok
  2    1     30%        0%   +30%  mismatch

At that exact moment, the rear fan of GPU 2 was physically spinning at 100%, even though the NVML feedback tool reported 0% feedback for GPU 2 / fan 1.

Current load snapshot during the issue

Command:

nvidia-smi --query-gpu=index,uuid,temperature.gpu,pstate,power.draw,fan.speed,utilization.gpu --format=csv; nvml-fan-feedback

Output:

index, uuid, temperature.gpu, pstate, power.draw [W], fan.speed [%], utilization.gpu [%]
0, GPU-672ea43d-ba25-9650-b494-9787bd455159, 43, P8, 26.42 W, 0 %, 0 %
1, GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf, 61, P2, 210.81 W, 30 %, 41 %
2, GPU-5f33c236-dca2-c873-2886-c0e2aa54a727, 52, P2, 223.99 W, 30 %, 55 %
GPU  FAN  TARGET  FEEDBACK  DELTA  STATUS
---  ---  ------  --------  -----  --------
  1    0     30%       30%    +0%  ok
  1    1     30%       30%    +0%  ok
  2    0     30%       30%    +0%  ok
  2    1     30%        0%   +30%  mismatch

Again, at that moment, the rear fan of GPU 2 was physically at 100%.

Additional note

This same rear fan had already shown a previous issue. After physically opening the GPU and reseating/repositioning the fan cable, the behavior became normal again for a while.

Now the problem appears intermittently depending on reboot.

What I am trying to understand

Based on the data above:

1. Has anyone seen this specific RTX 3090 behavior before?
2. Can an RTX 3090 fan physically run at 100% while NVML reports 0% feedback for that individual fan?
3. Could this point to a fan cable / connector / PWM / tach feedback issue?
4. Could this be related to a GPU-side fan controller or fan header issue?
5. Are there known RTX 3090-specific fan harness or rear fan failure modes that match this behavior?
6. What additional diagnostics would be useful before replacing the fan assembly or inspecting the GPU PCB side?

I am deliberately avoiding assumptions here and would appreciate feedback from people who have direct RTX 3090 repair or fan-control experience.

[Sylva1n]

The machine is running since 4 hours. No reboot. But the noisy fan suddenly rotates at 41%.

❯ nvidia-smi --query-gpu=index,uuid,temperature.gpu,pstate,power.draw,fan.speed,utilization.gpu --format=csv; nvml-fan-feedback
index, uuid, temperature.gpu, pstate, power.draw [W], fan.speed [%], utilization.gpu [%]
0, GPU-672ea43d-ba25-9650-b494-9787bd455159, 48, P8, 26.81 W, 0 %, 0 %
1, GPU-17735f6e-cbc4-b01f-a1f7-08f0509705cf, 68, P2, 211.41 W, 49 %, 36 %
2, GPU-5f33c236-dca2-c873-2886-c0e2aa54a727, 65, P2, 223.96 W, 41 %, 54 %
GPU  FAN  TARGET  FEEDBACK  DELTA  STATUS
---  ---  ------  --------  -----  --------
  1    0     30%       49%   -19%  mismatch
  1    1     30%       49%   -19%  mismatch
  2    0     30%       41%   -11%  mismatch
  2    1     30%       41%   -11%  mismatch

[noonghunna]

Yeah — this is a known 3090 failure mode, and the 0%-feedback / 100%-physical / 30%-target combo is the tell. A model that fits every symptom you listed:

NVML "target" ≠ what the fan actually does. NVML reports the driver's requested duty (your 30%); the actual PWM is set by the card's onboard fan controller, which closes a loop on each fan's tach (RPM-sense) signal. When that controller stops seeing a valid tach pulse, most 3090 AIB controllers treat it as "fan stalled" and ramp that fan to 100% as a thermal fail-safe — exactly "target 30%, feedback 0%, physical 100%."

So this almost certainly isn't software or even the controller — it's an intermittent tach/connector fault on that specific rear fan. Two things confirm it: it needed reseating after you opened the card, and it recovers (your 4-hr "settled to 41%" snapshot). A dead controller wouldn't intermittently close the loop again — intermittent = marginal connector or a nicked sense wire.

Isolate it before touching the PCB:

1. Power off, reseat the rear fan connector (verify the latch clicks); inspect under the shroud for a backed-out pin or a pinched/nicked cable at the strain relief — the thin tach wire is the usual offender on cards opened for a repaste.
2. Swap test — move the suspect fan to a known-good header (or a good fan onto the suspect header). Follows the fan/cable → replace the fan; follows the header → card-side.
3. Force a fixed duty via coolbits (nvidia-settings -a GPUFanControlState=1 -a [fan:N]/GPUTargetFanSpeed=40) and watch nvidia-smi -q -d FAN per-fan. If the physical fan ignores a low fixed target, the controller→fan link is the fault.
4. Log fan + temp over time to rule out a legit thermal ramp (it isn't, given the 0% feedback — but worth excluding).

PCB/controller replacement is a last resort; the vast majority resolve at the connector or fan cable. Curious what the swap test shows.

[tomByrer]

Maybe repaste the heatsink? Also look into adding shims to help heatsink the VRAM, adding thermal putty to the capacitors, termal taping a small heatsink to the back, just slapping a case fan directly on the GPU...
I've done some of these, seems to help slightly.