Repository navigation
Conversation
This reverts commit a562c8a.
There was a problem hiding this comment.
Summary of Changes
Hello @zhyncs, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request serves as a complete revert of a previous change that aimed to introduce multi-architecture support for ROCm within the Docker build environment. By undoing those modifications, the build system for ROCm is streamlined back to a single-architecture approach, simplifying the Dockerfile, adjusting related Python dependencies, and updating AITER integration points to align with the reverted state.
Highlights
- Revert Multi-architecture Support: This pull request fully reverts the changes introduced in commit
a562c8a35c93d70374e2d3b57c12f66718113f17, which had added multi-architecture build support for ROCm to theDockerfile.rocm. - Dockerfile Simplification: The
docker/Dockerfile.rocmhas been significantly simplified by removing the conditional build logic for different GPU architectures (e.g.,gfx942,gfx950), consolidating the build process to a single base image and flow. This includes removing specific VLLM build steps and streamlining Triton and AITER installations. - Dependency Adjustment: The
python/pyproject.tomlfile has been updated to explicitly includevllm==0.6.7.dev2as a required dependency for thesrt_hipconfiguration. - AITER Integration Update: Changes in
python/sglang/srt/layers/quantization/fp8_utils.pyreflect an update in the AITER library integration, specifically renaming thegemm_a8w8_blockscalefunction togemm_a8w8_blockscale_CK(lines 45, 277). The AITER commit version in the Dockerfile has also been reverted tov0.1.3.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request reverts a previous commit that added multi-arch support for ROCm. The changes primarily affect the ROCm Dockerfile, restoring it to a single-architecture build process. I've identified a potential issue in the Dockerfile where a git checkout command was not correctly restored, which could affect build correctness. I've also suggested several improvements for consistency in the Dockerfile, such as using python3 and python3 -m pip uniformly. Finally, I've flagged a development dependency in pyproject.toml that might cause installation issues for users outside the specific build environment. Overall, the revert seems to achieve its goal, but the identified issues should be addressed to ensure the stability and clarity of the codebase.
| && if [ "${SGL_BRANCH}" = ${SGL_DEFAULT} ]; then \ | ||
| echo "Using ${SGL_DEFAULT}, default branch."; \ | ||
| git checkout ${SGL_DEFAULT}; \ | ||
| else \ | ||
| echo "Using ${SGL_BRANCH} branch."; \ | ||
| git checkout ${SGL_BRANCH}; \ | ||
| fi \ |
There was a problem hiding this comment.
The if block is missing the git checkout ${SGL_DEFAULT}; command. This was present in the code being reverted to. While git clone typically checks out the default branch, being explicit here improves robustness and makes it consistent with the else block.
&& if [ "${SGL_BRANCH}" = "${SGL_DEFAULT}" ]; then \
echo "Using ${SGL_DEFAULT}, default branch."; \
git checkout ${SGL_DEFAULT}; \
else \
echo "Using ${SGL_BRANCH} branch."; \
git checkout ${SGL_BRANCH}; \
fi \
| srt_hip = [ | ||
| "sglang[runtime_common]", | ||
| "torch", | ||
| "vllm==0.6.7.dev2", |
There was a problem hiding this comment.
Adding a dependency with a .dev version like vllm==0.6.7.dev2 can cause issues if it's not available on a public package index. The comment on line 78 suggests this is not from a public wheel, which could lead to installation failures for users who are not using the specific Docker build environment.
If this dependency is expected to be pre-installed in the environment (e.g., within the Docker image), declaring it here could be problematic. It might be better to handle this dependency as part of the environment setup and document it, rather than listing it as a package to be installed by pip.
| && rm -f pyproject.toml \ | ||
| && mv pyproject_rocm.toml pyproject.toml \ | ||
| && AMDGPU_TARGET=$GPU_ARCH_LIST python setup_rocm.py install \ | ||
| && python setup_rocm.py install \ |
| && if [ "$BUILD_TYPE" = "srt" ]; then \ | ||
| python -m pip --no-cache-dir install -e "python[srt_hip]"; \ | ||
| else \ | ||
| python -m pip --no-cache-dir install -e "python[all_hip]"; \ | ||
| fi |
| fi | ||
|
|
||
| RUN cp -r /sgl-workspace/sglang /sglang | ||
| RUN python -m pip cache purge |
| RUN pip install IPython \ | ||
| && pip install orjson \ | ||
| && pip install python-multipart \ | ||
| && pip install torchao \ | ||
| && pip install pybind11 |
There was a problem hiding this comment.
For consistency and to avoid potential issues with different python versions or pip installations, it's best practice to use python3 -m pip instead of just pip.
Additionally, combining these into a single pip install command is more efficient as it creates a single Docker layer and resolves dependencies once.
RUN python3 -m pip install \
IPython \
orjson \
python-multipart \
torchao \
pybind11
| && pip install torchao \ | ||
| && pip install pybind11 | ||
|
|
||
| RUN pip uninstall -y triton |
This reverts commit a562c8a.
Motivation
Modifications
Checklist