Skip to content

Commit ddce116

Browse files
authored
Fix for Top K disabling (ggml-org#480)
* Update gpttype_adapter.cpp * use n_vocab instead of 32000 for when top k is off
1 parent 8c6001d commit ddce116

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

gpttype_adapter.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1312,7 +1312,7 @@ generation_outputs gpttype_generate(const generation_inputs inputs, generation_o
13121312
}
13131313
if (params.top_k < 1)
13141314
{
1315-
params.top_k = 120; //to disable top_k we actually need to increase this value to a very high number
1315+
params.top_k = n_vocab; // all tokens in the vocabulary should be considered if top k is disabled
13161316
}
13171317
if (params.seed <= 0 || params.seed==0xFFFFFFFF)
13181318
{

0 commit comments

Comments
 (0)