Don't use cache when running tests

[bsipocz]

A few recent issues highlighted that the cache is used during testing. And that shouldn't be the case.
Currently, we applied local workarounds when it caused issues, but ideally, it should be fixed centrally in a way that the default cache is never used for the tests while maybe a temporary used when explicitly asked for, but never by default.

See a workaround for cached files for simbad: #2199 and linked issues therein, for atomic: #2215