feat: invariant reproducers are treated as invalid if minor modifications are made to source code

[0xalpharush]

If I just add a log to debug a failing invariant test, it triggers this. Ideally, the same txs would be run against the modified bytecode to see if it still fails.

foundry/crates/forge/src/runner.rs

Line 1062 in aa3cc0f

Failure from {:?} file was ignored because test contract bytecode has changed.",

[grandizzy]

@0xalpharush this was added to accommodate #9965 and to avoid confusion when test contract gets changed. Not sure what's the best solution here, maybe a new config to always run against the modified bytecode, wdyt?

[0xalpharush]

It'd be nice to automatically detect the sender, target, or selector is filtered out and maybe warn that the reproducers are invalid. The format could be the same as corpus and the reproducer treated as a regression test. Similar to proptest failures where you can store the failing case

[grandizzy]

yeah, they're already persisted in cache/invariant/failures/{InvariantContract}/{invariant_function} and replayed as a regression test before a new campaign starts, the only thing here is that if you change the test contract we warn and not replay

[0xalpharush]

The original issue seemed related to the test's settings changing and making the reproducer "invalid". Could the settings be used instead of the bytecode to invalidate the reproducer? E.g. if the reproducer contains a selector or sender that was added to a blacklist, it would be considered not eligible

[grandizzy]

this makes sense, so we can save the invariant settings (target contracts / selectors / senders) together with the reproduction sequence and skip if they differ from the current (instead recording and checking the bytecode)