Skip to content

Commit 51083d1

Browse files
authored
[docs] fix bug in deepspeed config (#36081)
bug fix
1 parent 02776d2 commit 51083d1

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/source/en/deepspeed.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1091,7 +1091,7 @@ ds_config = {
10911091
"overlap_comm": True,
10921092
"contiguous_gradients": True,
10931093
"reduce_bucket_size": model_hidden_size * model_hidden_size,
1094-
"stage3_prefetch_bucket_size": 0.9 * model_hidden_size * model_hidden_size,
1094+
"stage3_prefetch_bucket_size": 3774873,
10951095
"stage3_param_persistence_threshold": 10 * model_hidden_size
10961096
},
10971097
"steps_per_print": 2000,

0 commit comments

Comments
 (0)