Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
aa37261
Retry get_daily_ci_runs on stale GitHub API cache (#48374)
ydshieh Aug 27, 2026
8631167
Fix `safe_open` mmap memory exhaustion on Windows by using `pread` ba…
eryk-roch Aug 27, 2026
1987631
[ONNX] Skip affected models on torch 2.13 (two dynamo regressions) (#…
ydshieh Aug 27, 2026
155b899
[Qwen3VLMoe] Update `test_small_model_integration_test_batch` expecte…
ydshieh Aug 27, 2026
83d024e
[LongcatFlash] Fix test_longcat_generation_cpu: use device_map="cpu" …
ydshieh Aug 27, 2026
281dd53
fix: flash-attn fallback failing on torch2.13 (#48388)
NanoCode012 Aug 28, 2026
e972043
[docs] Pass ContinuousBatchingConfig and sliding window models (#48381)
stevhliu Aug 28, 2026
a19f0d7
Raise when a paged attention forward is called with no cache (#48297)
qgallouedec Aug 28, 2026
ba356d2
Bump transformers-mlinter to 0.1.5 and clear the new findings (#48259)
tarekziade Aug 28, 2026
a8e8d14
[CB] Fix wrong device scoping (#48370)
remi-or Aug 28, 2026
83d46aa
[Improvement] Make gated delta rule more explicit (#47625)
remi-or Aug 28, 2026
d4dc3f2
fix some failure in xpu (#48252)
sywangyi Aug 28, 2026
721e492
Update Tailscale action version in workflow (#48394)
glegendre01 Aug 28, 2026
9aafba6
skip mtp slow tests for now (#48328) (#48329)
tarekziade Aug 28, 2026
efed243
[VibeVoice] Skip generate export tests (flaky) (#48396)
ydshieh Aug 28, 2026
0b80c4a
Fix interval merge invariant in _find_disjoint (#47860)
sharmax-vikas Aug 28, 2026
5f8ab9b
Fix incorrect tuple return annotations on forward methods returning a…
Gronoxx Aug 28, 2026
805a9e9
Avoid print to stdout that fails the job `check_failed_tests` job (#4…
ydshieh Aug 28, 2026
42ca970
[Docs]: Update GLM 5.3 (#48401)
Dovis01 Aug 29, 2026
555fe99
Sync upstream transformers main into OMY
Qubitium Aug 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ssh-runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ jobs:
echo 'source /root/.env_setup' >> /root/.bashrc

- name: Tailscale # In order to be able to SSH when a test fails
uses: huggingface/tailscale-action@7d53c9737e53934c30290b5524d1c9b4a7c98c8a # main
uses: huggingface/tailscale-action@9d1a67c46875a956cf2d1975473da978a970a2a9 # main
with:
authkey: ${{ secrets.TAILSCALE_SSH_AUTHKEY }}
slackChannel: ${{ secrets.SLACK_CIFEEDBACK_CHANNEL }}
Expand Down
4 changes: 2 additions & 2 deletions docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -667,10 +667,10 @@
title: GLM-4.5, GLM-4.6, GLM-4.7
- local: model_doc/glm4_moe_lite
title: GLM-4.7-Flash
- local: model_doc/glm_moe_dsa
title: GLM-5, GLM-5.1, GLM-5.2, GLM-5.3
- local: model_doc/glm_image
title: GLM-Image
- local: model_doc/glm_moe_dsa
title: GlmMoeDsa
- local: model_doc/openai-gpt
title: GPT
- local: model_doc/gpt_neo
Expand Down
9 changes: 7 additions & 2 deletions docs/source/en/continuous_batching.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,10 @@ for chunk in manager.request_id_iter(request_id="streamed"):

## ContinuousBatchingConfig

[`ContinuousBatchingConfig`] controls the KV cache, scheduling, CUDA graphs, memory usage, and more. Pass it alongside [`GenerationConfig`] to customize continuous batching.
[`ContinuousBatchingConfig`] controls the KV cache, scheduling, CUDA graphs, memory usage, and more. Pass it to `generate_batch` or [`~ContinuousMixin.init_continuous_batching`] with the `continuous_batching_config` argument, separately from [`GenerationConfig`]. The two configs describe different things. [`GenerationConfig`] is model-centered and holds sampling and stopping parameters, while [`ContinuousBatchingConfig`] is hardware-centered and holds memory and scheduling parameters.

> [!WARNING]
> Setting `continuous_batching_config` on a [`GenerationConfig`] is deprecated, emits a `FutureWarning`, and will be removed in v5.19.

By default, `max_batch_tokens` is `8192`, bounded by available GPU memory and never below `256`, while `num_blocks` fills the remaining memory. Use the table below to help you pick the appropriate features.

Expand Down Expand Up @@ -358,6 +361,8 @@ The fast path relies on the `flash_attn_with_kvcache` kernel, which is available

For any other combination, or when the kernel can't be imported, the manager falls back to the varlen path. It logs a warning only when you set `max_blocks_per_request` explicitly.

Sliding window attention doesn't support block tables, so the cache forces `max_blocks_per_request` to `0` for any model with sliding window layers, regardless of the attention implementation. If you set a nonzero value, it's overridden and the cache logs `Sliding window attention groups detected: disabling block table support.`

### CPU offloading

CPU offloading copies evicted KV cache blocks to a pre-allocated pinned CPU buffer when the GPU KV cache is full. After cache space becomes available, the manager copies the blocks back to the GPU and resumes the request without recomputing its prompt and generated tokens.
Expand Down Expand Up @@ -470,7 +475,7 @@ model = AutoModelForCausalLM.from_pretrained(
)
```

Prefix caching is disabled automatically when sliding window attention is active.
Prefix caching and the [decode fast path](#decode-fast-path) are disabled automatically when sliding window attention is active.

## Next steps

Expand Down
14 changes: 12 additions & 2 deletions docs/source/en/model_doc/glm_moe_dsa.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,21 @@ limitations under the License.
</div>
</div>

# GLM-5, GLM-5.1, GLM-5.2
# GLM-5, GLM-5.1, GLM-5.2, GLM-5.3

## Overview

**GLM-5**, **GLM-5.1** and **GLM-5.2** language model use this class. The implementation in transformers does not include an MTP layer.
**GLM-5**, **GLM-5.1**, **GLM-5.2** and **GLM-5.3** language model use this class. The implementation in transformers does not include an MTP layer.

### GLM-5.3

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks:

+ Stronger Coding: GLM-5.3 is the most capable open-weights model for coding, with a 50% improvement over GLM-5.2 on our in-house Z.ai Code Bench. It also achieve open-source SOTA on public benchmarks including Terminal Bench 3.0 and
Agents' Last Exam.
+ Emergent Cyber Capability: As we scaled post-training, cyber capability developed faster than we expected. GLM-5.3 is state of the art on CyberGym for vulnerability discovery, and its gains are largest further up the exploitation chain, where it more than doubles GLM-5.2 on exploitation benchmarks.

![bench_53](https://raw.githubusercontent.com/zai-org/GLM-5/refs/heads/main/resources/bench_53_2.png)

### GLM-5.2

Expand Down
Loading
Loading