Skip to content

[ut] make test can pass when LLM answer Chinese - #4346

Closed
panpan0000 wants to merge 1 commit into
sgl-project:mainfrom
panpan0000:refine-paris-test
Closed

panpan0000 wants to merge 1 commit into
sgl-project:mainfrom
panpan0000:refine-paris-test

Conversation

@panpan0000

Copy link
Copy Markdown
Contributor

Motivation

Make the UT more robust

Because the model is Qwen (Qwen1.5-1.8B-Chat-GPTQ-4bits-dynamic-cfg-with-lm_head-symFalse)
it will answer Chinese for question The capital of France is sometimes.

Failure case:
image

Traceback (most recent call last):
  File "/public_sglang_ci/runner-a-gpu-67/_work/sglang/sglang/test/srt/test_gptqmodel_dynamic.py", line 201, in test_throughput
    assert "paris" in result["text"].lower()
AssertionError

----------------------------------------------------------------------
Ran 4 tests in 72.148s

FAILED (failures=1)
result = `{'text': ' ______________. November 23rd\nB. The Summer Palace C. Shanghai\n\n答案:D  根据常识,法国的首都是巴黎,正确的选项应是D。故选D。第11题 南方电网公司《安全工作规定》的调度口号是“       安全第一,预防为主。”()  A.预防、精心、规范、协同、稳定 B.预防、精心、规范、提高、稳定 C.预防、精心、协同、定制、稳定 D.预防、精心、规范、协同、牢固 第12题 下列有关“加强班组基础工作”的说法中错误的是()  A.“加强班组基础工作”是第3次全国职工安全生活大讨论的主题 B.“加强班组基础工作”要求各级调度机构和变电站全面建立闭环管理机制 C.“加强班组基础工作”要求各级调度机构和变电站熟练掌握电力工作票制度 D.“加强班组基础工作”要求各级调度机构和变电站加强变电站内的安全管理 第13题 电力调度员的职业道德规范主要体现在四个方面,即()  A.对国家和人民负责 B.履行法定权利 C.承担严峻风险 D', 'meta_info': {'id': '6959c9725a2d4d4a8c66bdca8ba6a7fb', 'finish_reason': {'type': 'length', 'length': 256}, 'prompt_tokens': 5, 'completion_tokens': 256, 'cached_tokens': 2, 'e2e_latency': 0.5888371467590332}}`

Modifications

Checklist

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
@merrymercy

Copy link
Copy Markdown
Collaborator

Can you set temp=0?

@merrymercy

Copy link
Copy Markdown
Collaborator

I will set temp = 0 in my other PRs. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants