You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Import Errors occurring even when dependencies are installed #403
I am unable to create a Runtime with sglang as follows runtime = sgl.Runtime(model_path=MODEL_DIR, tokenizer_path=MODEL_DIR). It throws the error below:
I am running this on an A100 with the Nvidia PyTorch base image recommended by the Flash Attention repo ("nvcr.io/nvidia/pytorch:24.03-py3"). I have confirmed via print statements that all the dependencies are installing in the desired location and the files are actually there. I have tried installing sglang both with pip and from source and get the exact same error.
I noticed that not every package in the file structure has an __init__.py file, which can cause issues. Perhaps the version of PyTorch in the base image conflicts somehow with what sglang expects? Aside from that, I am not sure why this is happening. But given how simple this code is, I suspect there is an issue with the library, not the machine.
Activity
nikita-lukashevich1 commentedon May 1, 2024
Look here: #391
david-vectorflow commentedon May 1, 2024
Thanks! Looks like it still isn't solved though?
github-actions commentedon Jul 25, 2024
This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.
Merge pull request sgl-project#403 from shun8686/xyf-mtp-final
[sgl-project#673] Inline FLA's CHUNK_SIZE; import-weight pins and the…
[sgl-project#673][sgl-project#237] Move the triton monkeypatch into t…