How much memory is needed to train Eagle3 for Qwen3-32b?

[IdoCohenTD]

I am using the online training version, and I have the target and draft on two different gpus. How much memory did you need to run your default params (batch of 8, max_len of 2048)?

[liusong1222]

About 104 GB of GPU memory per card would be needed when running the online training version with our default parameters. Note that in our online training setup, the target and draft are both on the same GPU.

[IdoCohenTD]

I changed the script slightly so that the target model is sharded between two gpus, and the draft has its own gpu. Running like this on 3 A100 with 40G I still get OOM for the draft gpu with batch 1 and model_len 2048.