Skip to content

Fix the JVP of pivoted QR for wide matrices - #42634

Open
bhaskargurram-ai wants to merge 1 commit into
jax-ml:mainfrom
bhaskargurram-ai:fix/qr-pivoting-wide-jvp
Open

bhaskargurram-ai wants to merge 1 commit into
jax-ml:mainfrom
bhaskargurram-ai:fix/qr-pivoting-wide-jvp

Conversation

@bhaskargurram-ai

Copy link
Copy Markdown
Contributor

Fixes #42633

With pivoting=True and a wide input (m < n), the JVP of the R factor from lax.linalg.qr / jax.scipy.linalg.qr was wrong for the trailing n - m columns, so gradients through R were silently incorrect.

The wide-case rule computes dR2 = dQ^H X2 + Q^H dX2. With pivoting Q R = X P, and the rule already permutes dx, but X2 was taken from the unpermuted x. This applies the same permutation to x.

Test: testScipyQrModes only ran its pivoted JVP check for square and tall economic shapes. It now runs for every pivoting mode except mode='r' (which has no Q). The new wide cases fail on main and pass with this change; the tall mode='full' cases already passed.

For m < n the JVP computes dR2 = dQ^H X2 + Q^H dX2, where X2 holds the
trailing columns of X. With pivoting, Q R = X P, so X2 must come from
the permuted matrix like dX does. Using the unpermuted X gave wrong
derivatives for the trailing columns of R.

Also run the JVP check in testScipyQrModes for all pivoting cases
except mode='r'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong JVP/gradient of jax.scipy.linalg.qr(..., pivoting=True) for wide matrices

1 participant