Skip to content

[CANN]Support OP MUL_MAT_ID #13042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[CANN]Support OP MUL_MAT_ID #13042

wants to merge 2 commits into from
+186 −2

Conversation

noemotiovon
Copy link
Contributor

@noemotiovon noemotiovon commented Apr 21, 2025 •
edited
Loading

Why is this PR needed?

Add support for the MUL_MAT_ID operator required by MOE models.

Op Test

Backend 1/2: CANN0
ggml_backend_cann_context: device 0 async operator submission is OFF
  Device description: Ascend910B3
  Device memory: 62432 MB (62145 MB free)

  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): new_pool_for_device: device 0 use vmm pool
OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F32,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): OK
  MUL_MAT_ID(type_a=F16,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): OK
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_0,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=1,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=2,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=0,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=8,n_used=4,b=1,m=512,n=129,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q4_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q5_1,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=Q8_0,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=1,k=256): not supported [CANN0] 
  MUL_MAT_ID(type_a=UNKNOWN,type_b=F32,n_mats=4,n_used=2,b=0,m=512,n=32,k=256): not supported [CANN0] 
  5473/5473 tests passed
  Backend CANN0: OK

Backend 2/2: CPU
  Skipping
2/2 backends passed
OK

TODO

The current matrix multiplication in MOE is quite slow. I’ll keep investing effort into this and look for a suitable aclnn acceleration operator as a replacement.

noemotiovon added 2 commits April 21, 2025 07:52
[CANN]Support OP MUL_MAT_ID
ca501fb 
Signed-off-by: noemotiovon <[email protected]>
[CANN]Remove Chinese comments
1d3be02 
Signed-off-by: noemotiovon <[email protected]>
@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Apr 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@noemotiovon