Bug Report: 【FR】attention/rms_norm/silu/rotary operators need reference routing for accuracy on nvidia (openbmb/MiniCPM4.1-8B)
Description
On model openbmb/MiniCPM4.1-8B under FlagOS, four operators had insufficient FlagOS
accuracy. After routing them to reference (via nvidia.yaml op_backends), V3 GPQA = 58.0%,
above the NV baseline 54.0% (rel_drop -7.41%), passed.
Note: operator control in plugin mode is via vllm-plugin-FL nvidia.yaml op_backends, not gems.txt.
Environment
| Item |
Value |
| Hardware |
NVIDIA H20-3e |
| PyTorch |
2.11.0+cu130 |
| vLLM |
0.20.2 |
| FlagGems |
5.0.2 |
| vllm-plugin-FL |
0.2.0rc2.post1 |
Accuracy Data
- First probe 48% (temp=0.6 chain-of-thought noise) -> retest V3 = 58%, passed
- Fixed config: 4 operators dispatch -> reference first
Affected Operators
attention_backend, rms_norm, silu_and_mul, rotary_embedding
(FlagOS implementation accuracy insufficient; passes after routing to reference)
Status
- Resolved via nvidia.yaml dispatch tuning; V3 passed and released.
- Filed as an accuracy-improvement record for these 4 FlagGems operators; may be closed
once FlagGems side confirms.
Possible Directions
- Review FlagOS precision of rms_norm / rotary_embedding / silu_and_mul / attention
Bug Report: 【FR】attention/rms_norm/silu/rotary operators need reference routing for accuracy on nvidia (openbmb/MiniCPM4.1-8B)
Description
On model openbmb/MiniCPM4.1-8B under FlagOS, four operators had insufficient FlagOS
accuracy. After routing them to reference (via
nvidia.yaml op_backends), V3 GPQA = 58.0%,above the NV baseline 54.0% (rel_drop -7.41%), passed.
Environment
Accuracy Data
Affected Operators
attention_backend, rms_norm, silu_and_mul, rotary_embedding
(FlagOS implementation accuracy insufficient; passes after routing to reference)
Status
once FlagGems side confirms.
Possible Directions