Env-specific tests fail depending on random execution order #224
Description
Bug Report
| Q | A |
|---|---|
| Version(s) | 1.20.0 |
#220 introduced new environment variables, but the tests around it fail due to missing isolation of global state during test execution.
See for example #223 failure @ https://github.com/laminas/automatic-releases/actions/runs/3716382625/jobs/6302643279#step:3:512
Error: ] Project tests must be in a passing state before running Infection.
Infection runs the test suite in a RANDOM order. Make sure your tests
do not have hidden dependencies.
You can add these attributes to `phpunit.xml` to check it: <phpunit
executionOrder="random" resolveDependencies="true" ...
If you don't want to let Infection run tests in a random order, set the
`executionOrder` to some value, for example <phpunit
executionOrder="default"
Check the executed command to identify the problem:
'/usr/bin/phpdbg8.1' '-qrr' '/github/workspace/vendor/bin/phpunit'
'--configuration'
'/tmp/infection/phpunitConfiguration.initial.infection.xml'
'--coverage-xml=/tmp/infection/coverage-xml'
'--log-junit=/tmp/infection/junit.xml'
PHPUnit reported an exit code of 1.
Refer to the PHPUnit's output below:
STDOUT:
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.
Runtime: PHPDBG 8.1.13
Configuration:
/tmp/infection/phpunitConfiguration.initial.infection.xml
Random Seed: 1671223208
............................................................... 63 /
229 ( 27%)
........................F
Time: 00:02.168, Memory: 46.00 MB
There was 1 failure:
1)
Laminas\AutomaticReleases\Test\Unit\Environment\EnvironmentVariablesTes
t::testDebugModeOffEnvironmentVariables
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'INFO'
+'DEBUG'
/github/workspace/test/unit/Environment/EnvironmentVariablesTest.php:13
1
FAILURES!
Tests: 88, Assertions: 296, Failures: 1.
Generating code coverage report in PHPUnit XML format ... done
[00:00.288] \
Should be reproducible with the same random seed.