Skip to content

CANN: update docker images to 8.5.0 and improve CANN.md#20801

Merged
hipudding merged 4 commits into
ggml-org:masterfrom
KokerZhou:cann-pr
Mar 27, 2026
Merged

CANN: update docker images to 8.5.0 and improve CANN.md#20801
hipudding merged 4 commits into
ggml-org:masterfrom
KokerZhou:cann-pr

Conversation

@KokerZhou

Copy link
Copy Markdown
Contributor

This PR includes small updates to CANN-related components:

  • update docker images from rc versions to stable 8.5.0

    • this helps ensure a more reliable environment and enables further optimization using newer ops available in stable releases
  • update CANN.md to clarify device naming

    • the original document refers to "Atlas 300T A2"
    • however, only "Atlas 300I A2" appears to be officially available
    • the documentation has been adjusted accordingly

These changes are limited to CANN environment setup and documentation.
No functional behavior outside of CANN is affected.

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.
@KokerZhou KokerZhou requested review from a team and ngxson as code owners March 20, 2026 14:32
@CISC CISC requested a review from a team March 20, 2026 14:33
@github-actions github-actions Bot added documentation Improvements or additions to documentation devops improvements to build systems and github actions Ascend NPU issues specific to Ascend NPUs labels Mar 20, 2026
@hipudding

Copy link
Copy Markdown
Contributor

Thank you very much for your submission. This is something we've been wanting to do recently. llama.cpp only verifies the latest CAN version, and timely updates to the CAN version will result in better compatibility.

@hipudding hipudding self-requested a review March 23, 2026 06:23
@hipudding hipudding changed the title cann: update docker images to 8.5.0 and improve CANN.md CANN: update docker images to 8.5.0 and improve CANN.md Mar 23, 2026
@hipudding

Copy link
Copy Markdown
Contributor

BTW. bf16 is supported by #20152, could you please also add this data type in CANN.md? Thanks.

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.
@KokerZhou

Copy link
Copy Markdown
Contributor Author

Clarified BF16 support for 910B. Leave it empty for 310P.
On 310P, BF16/Q8_0/Q4_0 run but may be slower than FP16, as they do not leverage NPU acceleration (likely CPU-backed), as reported in #18797.
BTW, it seems that aclnnWeightQuantBatchMatmulV2 can now work on 310P under certain conditions. The issue mentioned there can be handled carefully later.

@hipudding

hipudding commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Clarified BF16 support for 910B. Leave it empty for 310P. On 310P, BF16/Q8_0/Q4_0 run but may be slower than FP16, as they do not leverage NPU acceleration (likely CPU-backed), as reported in #18797. BTW, it seems that aclnnWeightQuantBatchMatmulV2 can now work on 310P under certain conditions. The issue mentioned there can be handled carefully later.

Thanks for your feedback. Q8 and Q4 are slow even it run on NPU, not recommend to use 310p with these dtypes.

Fix 234: Trailing whitespace
Comment thread .devops/llama-cli-cann.Dockerfile
@hipudding hipudding merged commit 6861f65 into ggml-org:master Mar 27, 2026
8 checks passed
slartibardfast pushed a commit to slartibardfast/llama.cpp that referenced this pull request Apr 12, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
* cann: update docker images to 8.5.0

- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0

Move to newer stable releases.

* cann: update CANN.md

* Update CANN.md to include BF16 support

Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.

* Fix formatting issues in CANN.md

Fix 234: Trailing whitespace
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ascend NPU issues specific to Ascend NPUs devops improvements to build systems and github actions documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants