You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Installing sglang[openai] does not have numpy requirement #70
I tried installing sglang[openai], ran the code in the readme:
importsglangassglimportsnoop@sgl.functiondeftool_use(s, question):
s+="To answer this question: "+question+", "s+="I need to use a "+sgl.gen("tool", choices=["calculator", "web browser"]) +". "ifs["tool"] =="calculator":
s+="The math expression is"+sgl.gen("expression")
elifs["tool"] =="web browser":
s+="The website url is"+sgl.gen("url")
@sgl.functiondeftip_suggestion(s):
s+= (
"Here are two tips for staying healthy: ""1. Balanced Diet. 2. Regular Exercise.\n\n"
)
forks=s.fork(2)
fori, finenumerate(forks):
f+=f"Now, expand tip {i+1} into a paragraph:\n"f+=sgl.gen(f"detailed_tip", max_tokens=256, stop="\n\n")
s+="Tip 1:"+forks[0]["detailed_tip"] +"\n"s+="Tip 2:"+forks[1]["detailed_tip"] +"\n"s+="In summary"+sgl.gen("summary")
@sgl.functiondeftext_qa(s, question):
s+="Q: "+question+"\n"s+="A:"+sgl.gen("answer", stop="\n")
defdriver_tool_use():
state=tool_use.run(question="What is the capital of the United States?")
print(state.text())
print("\n")
defdriver_tip_suggestion():
state=tip_suggestion.run()
print(state.text())
print("\n")
defdriver_batching():
states=text_qa.run_batch(
[
{"question": "What is the capital of the United Kingdom?"},
{"question": "What is the capital of France?"},
{"question": "What is the capital of Japan?"},
],
progress_bar=True
)
forsinstates:
print(s.text())
print("\n")
defdriver_stream():
state=text_qa.run(
question="What is the capital of France?",
temperature=0.1,
stream=True
)
foroutinstate.text_iter():
print(out, end="", flush=True)
print("\n")
if__name__=="__main__":
sgl.set_default_backend(sgl.OpenAI("gpt-3.5-turbo-instruct"))
driver_tool_use()
driver_tip_suggestion()
driver_batching()
driver_stream()
Fix: Add numpy + possibly more dependencies to the SGlang dependencies.
Activity
merrymercy commentedon Jan 21, 2024
Thanks for reporting this. Fixed by #71
Support Demos (sgl-project#70)
Merge pull request sgl-project#70 from Furion-cn/refactor-more-metrics
[INT4] Add W4A8 MoE and frontend (sgl-project#70)
Resolve ReshapeAndCacheOperation setup error (sgl-project#70)
[Docs] Add traffic-scenario to github-pages (sgl-project#70)
CI: Test VL model on other Aiter repo (#70)
[NPU] optimize glm4.7 (sgl-project#70)
[GGUF] Shape-aware dispatch, capture-safe dequant chunking, auto reserve
benchmark+findings: the per-call read does not depend on how many lay…
Fix bugs that substitude get_world_group with get_parallel().world_gr…