debugNetworkImageHttpClientProvider does not work with setUp and tearDown

[tp]

I wanted to make use of the recently introduced debugNetworkImageHttpClientProvider in combination with the test's setUp and tearDown helpers. But sadly unlike debugDisableShadows from the same package, it doesn't work even when resetting to null.

Steps to Reproduce

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

Future<void> main() async {
  setUp(() {
    debugDisableShadows = true;
    debugNetworkImageHttpClientProvider = () => null;
  });

  tearDown(() {
    debugDisableShadows = false;
    debugNetworkImageHttpClientProvider = null;
  });

  testWidgets(
    'Empty test',
    (tester) async {
    },
  );
}

• Run flutter test

Logs

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
The value of a painting debug variable was changed by the test.

When the exception was thrown, this was the stack:
#0      debugAssertAllPaintingVarsUnset.<anonymous closure> (package:flutter/src/painting/debug.dart:47:7)
#1      debugAssertAllPaintingVarsUnset (package:flutter/src/painting/debug.dart:50:4)
#2      TestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart:661:12)
#3      AutomatedTestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart:1044:11)
#4      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:643:7)
<asynchronous suspension>
#7      TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:613:14)
#8      AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:1010:24)
#14     AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:1007:15)
#15     testWidgets.<anonymous closure> (package:flutter_test/src/widget_tester.dart:116:22)
#16     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:168:27)
<asynchronous suspension>
#17     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test_api/src/backend/invoker.dart:250:15)
<asynchronous suspension>
#22     Invoker.waitForOutstandingCallbacks (package:test_api/src/backend/invoker.dart:247:5)
#23     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:166:33)
#28     Declarer.test.<anonymous closure> (package:test_api/src/backend/declarer.dart:165:13)
<asynchronous suspension>
#29     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:400:25)
<asynchronous suspension>
#43     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:382:19)
#44     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:416:5)
#45     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)
(elided 28 frames from class _FakeAsync, package dart:async, package dart:async-patch, and package stack_trace)

The test description was:
  Empty test
════════════════════════════════════════════════════════════════════════════════════════════════════

No issues found! (ran in 1.5s)

[✓] Flutter (Channel stable, v1.7.8+hotfix.3, on Mac OS X 10.14.5 18F132, locale en-DE)
    • Flutter version 1.7.8+hotfix.3 at /Users/timm-l1-ay/Downloads/flutter
    • Framework revision b712a172f9 (9 days ago), 2019-07-09 13:14:38 -0700
    • Engine revision 54ad777fd2
    • Dart version 2.4.0

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/setup/#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, set ANDROID_HOME to that
      location.
      You may also want to add it to your PATH environment variable.


[✓] Xcode - develop for iOS and macOS (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    • CocoaPods version 1.7.2

[✓] iOS tools - develop for iOS devices
    • ios-deploy 1.9.4

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/setup/#android-setup for detailed instructions).

[!] IntelliJ IDEA Ultimate Edition (version 2018.2.6)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] VS Code
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.2.0

[✓] Connected device (1 available)
    • iPhone XS • 30D1E09F-587E-4525-847B-573F4B0E4420 • ios •
      com.apple.CoreSimulator.SimRuntime.iOS-12-1 (simulator)

! Doctor found issues in 3 categories.

[tp]

cc @tvolkert

[tp]

I guess as a workaround I could wrap the testWidgets's callback with something that adds the test HTTP client provider.

Just wondering how this behavior comes about. I feel either tearDown should run before the check and debugNetworkImageHttpClientProvider should pass, or if it runs later, then it should also fail for the overwritten debugDisableShadows.

Possibly related: #18119

[tvolkert]

@tp the way this is currently designed, setUp() and tearDown() are run outside the bounds of our controller widget tester environment and are thus not included in the post-test invariant checkers.

The fact that it debugDisableShadows appears to work is a red herring -- that property is getting overridden to true by the test framework itself, and thus it doesn't cause issues in your sample code because it leaves the test method (in between setUp() and tearDown()) in the desired state from the test framework's point of view.

The real solution here would be to include setUp() and tearDown() in the domain of the widget tester -- and ideally augmenting their API by having their signatures be setUp(WidgetTester) and tearDown(WidgetTester). But it's too late for such a change (except for maybe during a 2.0 version bump) -- it would be a massively breaking change.

In the meantime, closing this as working as intended. As you noticed, you can work around this by just updating the values in the body of the test instead of during setUp() and tearDown().

FYI @Hixie @goderbauer

[Hixie]

yet more reason to just dump these test features...

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.