hexagon: new vtcm layouts and improved pipelines for MUL_MAT, MUL_MAT_ID and FLASH_ATTN_EXT

[max-krasnyansky]

Overview

Another round of updates for the big three (MUL_MAT, MUL_MAT_ID, FLASH_ATTN_EXT) in ggml-hexagon.

This update introduces a concept of 'VTCM layout' that is used by the host CPU for selecting the right kernel (HMX, HVX, etc) and decide on the VTCM partitioning. The same functions are used by the NPU to do the actual VTCM partitioning. The new layouts improve buffer overlap/reuse in several cases and allows us to put more data into VTCM. Kernel param solvers have improved as well (more iterations and tuning are coming).
All big Ops have been updated to use this and the smaller Ops are next (I'm working on GEGLU and some other ones that started popping up in the profiles as we improve the big ones).

The PR also adds significant improvements to the pipelining (DMA, HVX-work, HMX-work,...).
At the Op level we see major (sometimes 2x) gains shaving milliseconds from big MUL_MATs and FAs.
It does not always translate to big gains at the model level but we do see a steady improvement (see some examples below).

It's been tested on all supported devices (ie hexagon v73...v81) and most popular models in Q4_0/1, Q8_0, MXFP4.
With these changes we handle longer contexts much better (I've been playing with 16K and 32K).
Also the improved latency on smaller models helps with the MTP Spec-Dec (i.e it makes the drafters much faster, will share some updates on that separately).

Additional information

Details

## gemma-4-E2B_q4_0

Ventuno-Q
  prompt eval time = 1446.51 ms / 786 tokens ( 1.84 ms per token,  543.38 tokens per second) (vs 536.00 master)
         eval time = 4195.98 ms /  63 runs   (66.60 ms per token,   15.01 tokens per second) (vs  15.31 master)


S24U
  prompt eval time =  762.84 ms / 741 tokens ( 1.03 ms per token,  971.37 tokens per second) (vs 873.28 master)
         eval time = 2680.14 ms /  63 runs   (42.54 ms per token,   23.51 tokens per second) (vs  22.37 master)


S25+
  prompt eval time =  485.52 ms / 741 tokens ( 0.66 ms per token, 1526.21 tokens per second) (vs 1488.15 master)
         eval time = 2336.63 ms /  63 runs   (37.09 ms per token,   26.96 tokens per second) (vs   27.10 master)


S26+
  prompt eval time =  375.56 ms / 741 tokens ( 0.51 ms per token, 1973.08 tokens per second) (vs 1837.38  master)
         eval time = 2040.19 ms /  63 runs   (32.38 ms per token,   30.88 tokens per second) (vs   31.76  master)


X2-Elite
  prompt eval time =  338.34 ms / 741 tokens ( 0.46 ms per token, 2190.13 tokens per second) (vs1966.10  master)
         eval time = 1383.50 ms /  63 runs   (21.96 ms per token,   45.54 tokens per second) (vs  43.00  master)


## Qwen3.5-2B-Q4_0.gguf

S24U
  prompt eval time = 1148.01 ms / 742 tokens ( 1.55 ms per token,  646.34 tokens per second) (vs 594.07 master)
         eval time = 2795.79 ms /  63 runs   (44.38 ms per token,   22.53 tokens per second) (vs  22.12 master)


S25+
  prompt eval time =  728.27 ms / 742 tokens ( 0.98 ms per token, 1018.85 tokens per second) (vs 984.92 master)
         eval time = 2216.01 ms /  63 runs   (35.17 ms per token,   28.43 tokens per second) (vs  28.23 master)


S26+
  prompt eval time =  569.41 ms / 742 tokens ( 0.77 ms per token, 1303.10 tokens per second) (vs 1300.80 master)
         eval time = 2005.88 ms /  63 runs   (31.84 ms per token,   31.41 tokens per second) (vs   31.26 master)


X2-Elite
  prompt eval time =  560.11 ms / 742 tokens ( 0.75 ms per token, 1324.74 tokens per second) (vs 1305.07  master)
         eval time = 1537.74 ms /  63 runs   (24.41 ms per token,   40.97 tokens per second) (vs   35.41  master)

## GPT-OSS-20B-mxfp4.gguf

X2-Elite
   prompt eval time = 1304.29 ms / 806 tokens ( 1.62 ms per token, 617.96 tokens per second)
          eval time = 3359.89 ms /  63 runs   (53.33 ms per token,  18.75 tokens per second)

## gemma-4-26B_q4_0-it.gguf

X2-Elite
   prompt eval time = 1678.44 ms / 741 tokens ( 2.27 ms per token, 441.48 tokens per second)
          eval time = 3896.27 ms /  63 runs   (61.85 ms per token,  16.17 tokens per second)

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes, the usual Antigravity assist for refactoring, profile analysis and drafting

[max-krasnyansky]

@ggml-org/maintainers may I get the second approval please.