Skip to content

mosaic gpu attention: unchecked shape values and missing test error #42614

Open

Description

@ndthr

The forward (https://github.com/jax-ml/jax/blob/main/jax/experimental/pallas/ops/gpu/attention_mgpu.py#L86-L89) and pipeline (https://github.com/jax-ml/jax/blob/main/jax/experimental/pallas/ops/gpu/attention_mgpu.py#L662-L665) attention implementations check the key tensor’s shape 2x instead of checking the value tensor which lets incorrectly shaped values pass

another bug:
The backwards test (https://github.com/jax-ml/jax/blob/main/tests/pallas/mgpu_attention_test.py#L179-L181) eats unrelated ValueErrors(unrelated to insufficient shmem), so the test can pass even if the actual computation fails.

Activity

  1. Andrewplaez commented on Nov 15, 2026

    @Andrewplaez

    I would like to work on the second bug If that is possible?

  2. Andrewplaez commented on Nov 15, 2026

    @Andrewplaez

    I just created a pr to fix the second issue. I am wondering if i can also submit one to work the first issue?

  3. ndthr commented on Nov 15, 2026

    @ndthr
    Author

    @Andrewplaez i think these r so small u can do both in one pr, and also i think its only rlly worth it if you test both on a gpu somehow. the 2nd one also has to be verified bc im not 200% sure.

  4. Andrewplaez commented on Nov 15, 2026

    @Andrewplaez

    Ok thanks for that, I will test them on a linux vm and pr both of them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions