You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
"- The user explicitly attempts to answer the question, AND\n"
295
295
f"- Their answer can be cleanly evaluated using: {eval_config.eval}\n"
296
-
"Ignore greetings, comments, non-answers, or requests for clarification."
296
+
"Ignore greetings, comments, non-answers, or requests for clarification.\n"
297
+
"Numerical word answers are allowed (e.g., 'five' is the same as '5').\n"
297
298
)
298
299
ifeval_config.eval_speaks_first:
299
-
system_prompt=f"You are an evaluation agent, be extremly brief. Numerical word answers are allowed. You will start the conversation by saying: '{example_prompt}'. {common_system_prompt}"
300
+
system_prompt=f"You are an evaluation agent, be extremly brief. You will start the conversation by saying: '{example_prompt}'. {common_system_prompt}"
300
301
else:
301
-
system_prompt=f"You are an evaluation agent, be extremly brief. Numerical word answers are allowed. First, ask one question: {example_prompt}. {common_system_prompt}"
302
+
system_prompt=f"You are an evaluation agent, be extremly brief. First, ask one question: {example_prompt}. {common_system_prompt}"
0 commit comments