Skip to content

[STORY-020] Verify both ornith models on the RTX 2060 (sm75) without the FA NaN crash #425

Description

@dogkeeper886

Context

Part of STORY-020 · plan #416

Goal

Verify both ornith models run coherently on the RTX 2060 (sm75) WITHOUT the flash-attention
NaN crash. Because ornith runs on the qwen35 arch (in the FA allowlist), FA enables on Turing —
so this run is only valid on an image that carries the #385/#395 Volta-only FA gate.

Done When

  • The sm75 run uses a gate-carrying image (confirmed in the prereq task); ornith does NOT
    panic: failed to sample token on Turing.
  • ornith:9b and ornith:35b pre-pulled on the sm75 runner.
  • test-models.yml run with runner_label=sm75; both models generate coherently.
  • Where ornith:35b exceeds the 2060's 6 GB, the offload split + throughput are reported
    as observed facts, not a correctness failure.
  • Results captured on this issue.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions