When I run alpaca_eval make_leaderboard, setting --is_return_instead_of_print to false results in the following error:
TypeError: print_leaderboard() got an unexpected keyword argument 'leaderboard_mode'
In main.py, utils.print_leaderboard is called with the argument leaderboard_mode=None.
However, in my environment this appears to be an unexpected keyword argument for print_leaderboard().
I suspect this is a naming mismatch: the correct keyword might be leaderboard_mode_or_models (or the function signature may have changed), so make_leaderboard is passing an outdated keyword name.
When I run
alpaca_eval make_leaderboard, setting--is_return_instead_of_printtofalseresults in the following error:In
main.py,utils.print_leaderboardis called with the argumentleaderboard_mode=None.However, in my environment this appears to be an unexpected keyword argument for
print_leaderboard().I suspect this is a naming mismatch: the correct keyword might be
leaderboard_mode_or_models(or the function signature may have changed), somake_leaderboardis passing an outdated keyword name.