Skip to content

Benchmark tests are failing #9674

@elliette

Description

@elliette

Our dart2wasm benchmark tests are failing with the following error (example run):

Can run web benchmarks with WASM (failed)
  Building Flutter web app (compiler: dart2wasm)...
  Build took 146s to complete.
  Launching Chrome.
  Launching Google Chrome 145.0.7632.76 
  
  Waiting for the benchmark to report benchmark profile.
  [CHROME]: 
  [CHROME]: DevTools listening on ws://127.0.0.1:10000/devtools/browser/3409c080-2f4f-4ae9-947d-b3de97f85223
  Connecting to DevTools: ws://localhost:10000/devtools/page/6A2B907AC5919DAE0A4AC907955D80F8
  Connected to Chrome tab:  (http://localhost:9999/?compiler=wasm)
  Launching benchmark "devtools_navigateThroughOfflineScreens"
  [APP] 2026-02-21 00:37:04.316: TEST STATUS: Warming up.
  [APP] 2026-02-21 00:37:04.428: TEST STATUS: Warm-up finished.
  [APP] 2026-02-21 00:37:04.429: TEST STATUS: Navigate through offline DevTools tabs
  [APP] 2026-02-21 00:37:04.431: TEST STATUS: switching to home screen (icon null, iconAsset: icons/app_bar/devtools.png)
  [APP] 2026-02-21 00:37:07.450: TEST STATUS: switching to performance screen (icon null, iconAsset: icons/app_bar/performance.png)
  [APP] 2026-02-21 00:37:10.485: TEST STATUS: switching to cpu-profiler screen (icon null, iconAsset: icons/app_bar/cpu_profiler.png)
  [APP] 2026-02-21 00:37:13.518: TEST STATUS: switching to memory screen (icon null, iconAsset: icons/app_bar/memory.png)
  [APP] 2026-02-21 00:37:16.537: TEST STATUS: switching to network screen (icon null, iconAsset: icons/app_bar/network.png)
  [APP] 2026-02-21 00:37:19.556: TEST STATUS: switching to app-size screen (icon null, iconAsset: icons/app_bar/app_size.png)
  [APP] 2026-02-21 00:37:22.589: TEST STATUS: switching to deep-links screen (icon null, iconAsset: icons/app_bar/deep_links.png)
  [APP] 2026-02-21 00:37:25.625: TEST STATUS: End navigate through offline DevTools tabs
  Extracted 1255 measured frames.
  Skipped 123 non-measured frames.
  [APP] Client preparing to reload the window to: "http://localhost:9999?compiler=wasm"
  Launching benchmark "devtools_offlineCpuProfilerScreen"
  [APP] 2026-02-21 00:37:31.227: TEST STATUS: Warming up.
  [APP] 2026-02-21 00:37:31.344: TEST STATUS: Warm-up finished.
  [APP] 2026-02-21 00:37:31.344: TEST STATUS: Loading offline CPU profiler data and interacting
  [APP] 2026-02-21 00:38:04.528: TEST STATUS: On Bottom Up tab by default. Scrolling through table.
  [APP] 2026-02-21 00:38:06.061: TEST STATUS: Switching to Call Tree tab.
  [APP] 2026-02-21 00:38:12.098: TEST STATUS: Scrolling through Call Tree table.
  [APP] 2026-02-21 00:38:13.648: TEST STATUS: Switching to Method Table tab.
  [APP] 2026-02-21 00:38:19.664: TEST STATUS: Scrolling through Method Table.
  [APP] 2026-02-21 00:38:21.198: TEST STATUS: Switching to CPU Flame Chart tab.
  Caught browser-side error: Exception: `startMeasureFrame` called twice in a row.
      at $.DomWindow|requestAnimationFrame (http://localhost:9999/main.dart.wasm:wasm-function[5969]:0x21c941)
      at $.EnginePlatformDispatcher.scheduleFrame (http://localhost:9999/main.dart.wasm:wasm-function[5967]:0x21c6a1)
      at $._RecordingWidgetsBinding.handleDrawFrame (http://localhost:9999/main.dart.wasm:wasm-function[29143]:0x50cdcc)
      at $._MixinApplication2&BindingBase&GestureBinding&SchedulerBinding.handleBeginFrame (http://localhost:9999/main.dart.wasm:wasm-function[11963]:0x2d431f)
      at $._RecordingWidgetsBinding.handleBeginFrame (http://localhost:9999/main.dart.wasm:wasm-function[29144]:0x50ceea)
      at $._MixinApplication2&BindingBase&GestureBinding&SchedulerBinding._handleBeginFrame tear-off trampoline (http://localhost:9999/main.dart.wasm:wasm-function[5974]:0x21ca75)
      at $.invoke1 (http://localhost:9999/main.dart.wasm:wasm-function[2066]:0x1aee6c)
      at $.closure wrapper at org-dartlang-sdk:///lib/_engine/engine/frame_service.dart:91:37 trampoline (http://localhost:9999/main.dart.wasm:wasm-function[5968]:0x21c7db)
  dart:async                                     _Completer.completeError
  package:web_benchmarks/src/runner.dart 254:25  BenchmarkServer.run.<fn>
  ===== asynchronous gap ===========================
  dart:async                                     _CustomZone.registerUnaryCallback
  package:web_benchmarks/src/runner.dart 248:27  BenchmarkServer.run.<fn>
  package:shelf/src/cascade.dart 77:53           Cascade.handler.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                     Future.then
  package:shelf/src/cascade.dart 76:58           Cascade.handler.<fn>
  package:shelf/src/cascade.dart 76:47           Cascade.handler.<fn>.<fn>
  dart:async                                     new Future.sync
  package:shelf/src/cascade.dart 76:21           Cascade.handler.<fn>
  package:shelf/shelf_io.dart 140:29             handleRequest
  package:shelf/shelf_io.dart 93:9               serveRequests.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:_http                                     _HttpServer.listen
  package:shelf/shelf_io.dart 92:14              serveRequests.<fn>
  package:shelf/src/util.dart 21:20              catchTopLevelErrors
  package:shelf/shelf_io.dart 91:3               serveRequests
  package:web_benchmarks/src/runner.dart 316:16  BenchmarkServer.run
  ===== asynchronous gap ===========================
  dart:async                                     _CustomZone.registerUnaryCallback
  package:web_benchmarks/src/runner.dart 136:42  BenchmarkServer.run
  package:web_benchmarks/server.dart 75:5        serveWebBenchmark
  benchmark/devtools_benchmarks_test.dart 57:28  _runBenchmarks
  ===== asynchronous gap ===========================
  dart:async                                     _CustomZone.registerUnaryCallback
  benchmark/devtools_benchmarks_test.dart 54:25  _runBenchmarks
  benchmark/devtools_benchmarks_test.dart 43:15  main.<fn>
Shell: Starting web benchmark tests ...
Shell: Web benchmark tests finished.
Shell: Verifying devtools_navigateThroughOfflineScreens.js scores against expected thresholds.
Shell: Verifying devtools_offlineCpuProfilerScreen.js scores against expected thresholds.
Shell: Verifying devtools_offlinePerformanceScreen.js scores against expected thresholds.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1high priority issues at the top of the work list, actively being worked on.broken testtesting

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions