Skip to content

autodiff fails when differentiating two identical functions #139471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ZuseZ4 opened this issue Apr 7, 2025 · 0 comments · Fixed by #139700
Closed

autodiff fails when differentiating two identical functions #139471

ZuseZ4 opened this issue Apr 7, 2025 · 0 comments · Fixed by #139700
Labels
C-bug Category: This is a bug. F-autodiff `#![feature(autodiff)]`

Comments

@ZuseZ4
Copy link
Member

ZuseZ4 commented Apr 7, 2025

I tried this code:

#![feature(autodiff)]

    use std::autodiff::autodiff;

    #[autodiff(d_square, Reverse, Duplicated, Active)]
    fn square(x: &f64) -> f64 {
        x * x
    }

    #[autodiff(d_square2, Reverse, Duplicated, Active)]
    fn square2(x: &f64) -> f64 {
        x * x
    }

fn main() {
    let x = 3.0;
    let mut dx = 1.0;
    d_square(&x, &mut dx, 1.0);
    d_square2(&x, &mut dx, 1.0);
}

I expected to see this happen: d_square and d_square2 should both compute the derivative of x*x, thus 2.0 * x. However, LLVM (or something earlier?) recognizes that square and square2 are identical, and fuses them (ReplaceAllUsesWith, or RAUW for LLVM). autodiff then is missing a (source) function to differentiate, giving the following error. This is often happening in our test cases, where we just test the same dummy functions in a lot of different configurations.

Instead, this happened:

error: failed to prepare autodiff: src: _ZN2ad7square217h920417192f1e5f52E, target: _ZN2ad9d_square217he1a6ffbcf6ea74f2E, could not find source function

Meta

rustc --version --verbose:

build from source
Backtrace

<backtrace>

Tracking:

@ZuseZ4 ZuseZ4 added the C-bug Category: This is a bug. label Apr 7, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 7, 2025
@ZuseZ4 ZuseZ4 added F-autodiff `#![feature(autodiff)]` and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Apr 7, 2025
bors added a commit to rust-lang-ci/rust that referenced this issue Apr 22, 2025
Autodiff flags

Interestingly, it seems that some other projects have conflicts with exactly the same LLVM optimization passes as autodiff.
At least `LLVMRustOptimize` has exactly the flags that we need to disable problematic opt passes.

This PR enables us to compile code where users differentiate two identical functions in the same module. This has been especially common in test cases, but it's not impossible to encounter in the wild.

It also enables two new flags for testing/debugging. I consider writing an MCP to upgrade PrintPasses to be a standalone -Z flag, since it is *not* the same as `-Z print-llvm-passes`, which IMHO gives less useful output. A discussion can be found here: [#t-compiler/llvm > Print llvm passes. @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187780-t-compiler.2Fllvm/topic/Print.20llvm.20passes.2E/near/511533038)

Finally, it improves `PrintModBefore` and `PrintModAfter`. They used to work reliable, but now we just schedule enzyme as part of an existing ModulePassManager (MPM). Since Enzyme is last in the MPM scheduling, PrintModBefore became very inaccurate. It used to print the input module, which we gave to the Enzyme and was great to create llvm-ir reproducer. However, lately the MPM would run the whole `default<O3>` pipeline, which heavily modifies the llvm module, before we pass it to Enzyme. That made it impossible to use the flag to create llvm-ir reproducers for Enzyme bugs. We now schedule a PrintModule pass just before Enzyme, solving this problem.

Based on the PrintPass output, it also _seems_ like changing `registerEnzymeAndPassPipeline(PB, true);` to `registerEnzymeAndPassPipeline(PB, false);` has no effect. In theory, the bool should tell Enzyme to schedule some helpful passes in the PassBuilder. However, since it doesn't do anything and I'm not 100% sure anymore on whether we really need it, I'll just disable it for now and postpone investigations.

r? `@oli-obk`

closes rust-lang#139471

Tracking:

- rust-lang#124509
Zalathar added a commit to Zalathar/rust that referenced this issue Apr 24, 2025
Autodiff flags

Interestingly, it seems that some other projects have conflicts with exactly the same LLVM optimization passes as autodiff.
At least `LLVMRustOptimize` has exactly the flags that we need to disable problematic opt passes.

This PR enables us to compile code where users differentiate two identical functions in the same module. This has been especially common in test cases, but it's not impossible to encounter in the wild.

It also enables two new flags for testing/debugging. I consider writing an MCP to upgrade PrintPasses to be a standalone -Z flag, since it is *not* the same as `-Z print-llvm-passes`, which IMHO gives less useful output. A discussion can be found here: [#t-compiler/llvm > Print llvm passes. @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187780-t-compiler.2Fllvm/topic/Print.20llvm.20passes.2E/near/511533038)

Finally, it improves `PrintModBefore` and `PrintModAfter`. They used to work reliable, but now we just schedule enzyme as part of an existing ModulePassManager (MPM). Since Enzyme is last in the MPM scheduling, PrintModBefore became very inaccurate. It used to print the input module, which we gave to the Enzyme and was great to create llvm-ir reproducer. However, lately the MPM would run the whole `default<O3>` pipeline, which heavily modifies the llvm module, before we pass it to Enzyme. That made it impossible to use the flag to create llvm-ir reproducers for Enzyme bugs. We now schedule a PrintModule pass just before Enzyme, solving this problem.

Based on the PrintPass output, it also _seems_ like changing `registerEnzymeAndPassPipeline(PB, true);` to `registerEnzymeAndPassPipeline(PB, false);` has no effect. In theory, the bool should tell Enzyme to schedule some helpful passes in the PassBuilder. However, since it doesn't do anything and I'm not 100% sure anymore on whether we really need it, I'll just disable it for now and postpone investigations.

r? `@oli-obk`

closes rust-lang#139471

Tracking:

- rust-lang#124509
@bors bors closed this as completed in c3f811f Apr 24, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 24, 2025
Rollup merge of rust-lang#139700 - EnzymeAD:autodiff-flags, r=oli-obk

Autodiff flags

Interestingly, it seems that some other projects have conflicts with exactly the same LLVM optimization passes as autodiff.
At least `LLVMRustOptimize` has exactly the flags that we need to disable problematic opt passes.

This PR enables us to compile code where users differentiate two identical functions in the same module. This has been especially common in test cases, but it's not impossible to encounter in the wild.

It also enables two new flags for testing/debugging. I consider writing an MCP to upgrade PrintPasses to be a standalone -Z flag, since it is *not* the same as `-Z print-llvm-passes`, which IMHO gives less useful output. A discussion can be found here: [#t-compiler/llvm > Print llvm passes. @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187780-t-compiler.2Fllvm/topic/Print.20llvm.20passes.2E/near/511533038)

Finally, it improves `PrintModBefore` and `PrintModAfter`. They used to work reliable, but now we just schedule enzyme as part of an existing ModulePassManager (MPM). Since Enzyme is last in the MPM scheduling, PrintModBefore became very inaccurate. It used to print the input module, which we gave to the Enzyme and was great to create llvm-ir reproducer. However, lately the MPM would run the whole `default<O3>` pipeline, which heavily modifies the llvm module, before we pass it to Enzyme. That made it impossible to use the flag to create llvm-ir reproducers for Enzyme bugs. We now schedule a PrintModule pass just before Enzyme, solving this problem.

Based on the PrintPass output, it also _seems_ like changing `registerEnzymeAndPassPipeline(PB, true);` to `registerEnzymeAndPassPipeline(PB, false);` has no effect. In theory, the bool should tell Enzyme to schedule some helpful passes in the PassBuilder. However, since it doesn't do anything and I'm not 100% sure anymore on whether we really need it, I'll just disable it for now and postpone investigations.

r? ``@oli-obk``

closes rust-lang#139471

Tracking:

- rust-lang#124509
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. F-autodiff `#![feature(autodiff)]`
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants