Integration tests create stub modules in app/code

[Vinai]

When even a single integration test is executed, the following files and directories are created:

app/code/Magento
├── TestModuleDirectoryZipCodes
│   ├── etc
│   │   ├── module.xml
│   │   └── zip_codes.xml
│   └── registration.php
├── TestModuleFakePaymentMethod
│   ├── Gateway
│   │   └── Command
│   │       └── DoNothingCommand.php
│   ├── etc
│   │   ├── config.xml
│   │   ├── di.xml
│   │   └── module.xml
│   └── registration.php
└── TestModuleSample
    ├── composer.json
    ├── etc
    │   └── module.xml
    └── registration.php

(Note: the module Magento_TestModuleFakePaymentMethod is not generated in Magento 2.2.0).
This is a problem because running integration tests should not affect the installed Magento instance.

Preconditions

Reproduced on fresh installations of Magento 2.2.0 and 2.2.2. I haven't tried on Magento 2.2.1.

Steps to reproduce

1. Install Magento, e.g. composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition . and so on.
2. Run all integration tests or create a custom module with nothing but a single integration test.
3. Run the test, e.g. /var/www/example/vendor/phpunit/phpunit/phpunit --configuration /var/www/example/dev/tests/integration/phpunit.xml /var/www/example/app/code/Example/Foo/Test/Integration

Expected result

1. Only the specified tests should run in isolation.
2. Potential production code should not be affected.

Actual result

The modules are created in app/code/Magento potentially affect production code and are created independently if they are used in any test.

It should be possible to create the modules under dev/tests/integration/tmp in the sandbox directory and register them in the test process with the component registrar instead, leaving the app/code directory unmodified.

[magento-engcom-team]

Hi @Vinai. Thank you for the report. We are moving your feature request to the special project. You can track this issue here: magento/community-features#38

[avstudnitz]

@magento-engcom-team This is not a feature request, but clearly a bug. Please reopen and fix.

[miguelbalparda]

Reopening this, sorry about that.

[avstudnitz]

The problem with this behavior: I can add app/code/Magento to my .gitignore , but even if I do, and I call bin/magento setup:upgrade some time after running the integration tests, the modules are being added to app/etc/config.php which is not in my .gitignore (and shouldn't be).