Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

[web:canvaskit] switch to temporary SkPaint objects #54818

Merged
merged 4 commits into from
Aug 30, 2024

Conversation

yjbanov
Copy link
Contributor

@yjbanov yjbanov commented Aug 27, 2024

Do not eagerly create an SkPaint object that's strongly referenced by CkPaint. Instead, when a Canvas.draw* is called, create a temporary SkPaint object, pass it to Skia, then immediately delete it. This way there are no persistent SkPaint handles lurking in the system that transitively hold onto expensive native resources.

Addresses the Paint issue in flutter/flutter#153678 in CanvasKit

Spot checking some benchmarks. Here's the effect of this PR on draw_rect_variable_paint. It's a bit of a stress test as it creates 300K distinct Paint objects to render 600 pictures (a typical ratio does not normally exceed ten paints to one picture). Even so, the effect of creating an extra SkPaint on every draw* command does not look significant. However, removing a dependency on the GC for 300K objects looks like a good trade-off.

Before

Allocation stats:

  Paint Created: 300000
  Paint Deleted: 300000
  Paint Leaked: 300000
  Picture Created: 600
  Picture Deleted: 599
  Picture Leaked: 599

Performance stats:

windowRenderDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
 | average: 4679.551020408163 μs
 | outlier average: 5100 μs
 | outlier/clean ratio: 1.0898481452084188x
 | noise: 3.11%

sceneBuildDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
 | average: 4689.765306122449 μs
 | outlier average: 5100 μs
 | outlier/clean ratio: 1.087474461321549x
 | noise: 3.19%

drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 8447.474226804125 μs
 | outlier average: 9332.666666666666 μs
 | outlier/clean ratio: 1.1047878236850721x
 | noise: 3.52%

After

Allocation stats:

  Picture Created: 600
  Picture Deleted: 599
  Picture Leaked: 599

Performance stats:

windowRenderDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 4780.40206185567 μs
 | outlier average: 5133.666666666667 μs
 | outlier/clean ratio: 1.0738985131877936x
 | noise: 2.70%

sceneBuildDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 4787.6082474226805 μs
 | outlier average: 5133.666666666667 μs
 | outlier/clean ratio: 1.0722821085936345x
 | noise: 2.72%

drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 8243.309278350516 μs
 | outlier average: 9033.333333333334 μs
 | outlier/clean ratio: 1.0958382159768851x
 | noise: 2.60%

Follow-up

A few benchmarks regressed: recordPaintCommands, draw_rect (with static Paint), and clipped_out_pictures. The first two were expected, since I'm literally moving the work of creating SkPaint to the draw* invocations. clipped_out_pictures uses a static Paint object, so again the work to create SkPaint moved into the measured section of the benchmark.

Of course, there's a real trade-off in that if the same Paint object is heavily reused we repeatedly pay the cost of reallocating SkPaint. This does not seem to be noticeable in real-world situations though. None of our macrobenchmarks that use real widgets (e.g. Material 3 benchmarks) felt the change.

Overall, the trade-off seems to be worth it for two reasons:

  • It does help keep memory usage lower.
  • It is consistent with the mobile engine. Developers expect memory usage characteristics to be consistent no matter which platform they run the app on.

That said, perhaps there's room for improvement. For example, we could use stack-based allocation for the temporary objects, which is something Skwasm can do (I'll test it when I make the same change in Skwasm). We could also introduce an arena for all frame-scoped native objects. This would improve over stack-based allocation in that the same object can be reused in multiple places in the same frame, and cleaned up after the frame is done, perhaps even asynchronously to let the frame present to screen without waiting for the clean-up.

@github-actions github-actions bot added the platform-web Code specifically for the web engine label Aug 27, 2024
// TODO(matanlurey): Web team, if important to optimize, could:
// 1. Add a `engine.renderer.copyPaint` method.
// 2. Use the below code as the default implementation.
// 3. Have renderer-specific implementations override with optimized code.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this comment because all the setters are cheap now.

Copy link
Contributor

@eyebrowsoffire eyebrowsoffire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had some driveby comments.

if (value is ui.ColorFilter) {
filter = createCkColorFilter(value as EngineColorFilter);
_imageFilter = createCkColorFilter(value as EngineColorFilter);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it would be more logical to do this at the point of converstion, rather than in the setter. This happening in the setter always felt kind of weird to me anyway...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! The actual conversion to SkImageFilter happens in CkManagedSkImageFilterConvertible.imageFilter used in toSkPaint() above. But now that I read the various implementations of CkManagedSkImageFilterConvertible.imageFilter, they all use UniqueRef, so they are subject to GC. I'm going to have to fix that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -22,115 +22,82 @@ import 'shader.dart';
/// This class is backed by a Skia object that must be explicitly
/// deleted to avoid a memory leak. This is done by extending [SkiaObject].
class CkPaint implements ui.Paint {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we will have to do something similar for skwasm, do you think it might be worthwhile to pull this out of the canvaskit-specific code and instead just make it a renderer-agnostic class that I could reuse in skwasm? And perhaps just making an extension method for toSkPaint or whatever?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a bad idea! I'm not going to do it in this PR, since CkPaint has direct dependencies on canvaskit_api.dart and many other Ck* classes. But it can be done as a follow-up when SkwasmPaint moves to this model. I filed flutter/flutter#154281 and will leave a TODO.

@flutter-dashboard
Copy link

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

Changes reported for pull request #54818 at sha 780c6fc

Copy link
Contributor

@eyebrowsoffire eyebrowsoffire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@flutter-dashboard
Copy link

Golden file changes are available for triage from new commit, Click here to view.

Changes reported for pull request #54818 at sha ae5bddf

@yjbanov yjbanov added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 30, 2024
@auto-submit auto-submit bot merged commit 2d56e44 into flutter:main Aug 30, 2024
30 checks passed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Aug 30, 2024
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Aug 30, 2024
…154459)

flutter/engine@4f1de98...2d56e44

2024-08-30 [email protected] [web:canvaskit] switch to temporary SkPaint objects (flutter/engine#54818)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Buchimi pushed a commit to Buchimi/flutter that referenced this pull request Sep 2, 2024
…lutter#154459)

flutter/engine@4f1de98...2d56e44

2024-08-30 [email protected] [web:canvaskit] switch to temporary SkPaint objects (flutter/engine#54818)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
auto-submit bot pushed a commit that referenced this pull request Sep 6, 2024
Same as #54818, but for Skwasm.

Addresses the `Paint` issue in flutter/flutter#153678 in Skwasm
auto-submit bot added a commit that referenced this pull request Sep 7, 2024
Reverts: #54917
Initiated by: jonahwilliams
Reason for reverting: failing on framework -> engine roll  https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20web_skwasm_tests_7_last/10571/overview
Original PR Author: yjbanov

Reviewed By: {eyebrowsoffire}

This change reverts the following previous change:
Same as #54818, but for Skwasm.

Addresses the `Paint` issue in flutter/flutter#153678 in Skwasm
jesswrd pushed a commit to jesswrd/engine that referenced this pull request Sep 11, 2024
Same as flutter#54818, but for Skwasm.

Addresses the `Paint` issue in flutter/flutter#153678 in Skwasm
jesswrd pushed a commit to jesswrd/engine that referenced this pull request Sep 11, 2024
…utter#55018)

Reverts: flutter#54917
Initiated by: jonahwilliams
Reason for reverting: failing on framework -> engine roll  https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20web_skwasm_tests_7_last/10571/overview
Original PR Author: yjbanov

Reviewed By: {eyebrowsoffire}

This change reverts the following previous change:
Same as flutter#54818, but for Skwasm.

Addresses the `Paint` issue in flutter/flutter#153678 in Skwasm
brandon-bethke-timu pushed a commit to timu-com/engine that referenced this pull request Oct 4, 2024
Do not eagerly create an `SkPaint` object that's strongly referenced by `CkPaint`. Instead, when a `Canvas.draw*` is called, create a temporary `SkPaint` object, pass it to Skia, then immediately delete it. This way there are no persistent `SkPaint` handles lurking in the system that transitively hold onto expensive native resources.

Addresses the `Paint` issue in flutter/flutter#153678 in CanvasKit

Spot checking some benchmarks. Here's the effect of this PR on `draw_rect_variable_paint`. It's a bit of a stress test as it creates 300K distinct `Paint` objects to render 600 pictures (a typical ratio does not normally exceed ten paints to one picture). Even so, the effect of creating an extra `SkPaint` on every `draw*` command does not look significant. However, removing a dependency on the GC for 300K objects looks like a good trade-off.

Allocation stats:

```
  Paint Created: 300000
  Paint Deleted: 300000
  Paint Leaked: 300000
  Picture Created: 600
  Picture Deleted: 599
  Picture Leaked: 599
```

Performance stats:

```
windowRenderDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
 | average: 4679.551020408163 μs
 | outlier average: 5100 μs
 | outlier/clean ratio: 1.0898481452084188x
 | noise: 3.11%

sceneBuildDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
 | average: 4689.765306122449 μs
 | outlier average: 5100 μs
 | outlier/clean ratio: 1.087474461321549x
 | noise: 3.19%

drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 8447.474226804125 μs
 | outlier average: 9332.666666666666 μs
 | outlier/clean ratio: 1.1047878236850721x
 | noise: 3.52%
```

Allocation stats:

```
  Picture Created: 600
  Picture Deleted: 599
  Picture Leaked: 599
```

Performance stats:

```
windowRenderDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 4780.40206185567 μs
 | outlier average: 5133.666666666667 μs
 | outlier/clean ratio: 1.0738985131877936x
 | noise: 2.70%

sceneBuildDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 4787.6082474226805 μs
 | outlier average: 5133.666666666667 μs
 | outlier/clean ratio: 1.0722821085936345x
 | noise: 2.72%

drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
 | average: 8243.309278350516 μs
 | outlier average: 9033.333333333334 μs
 | outlier/clean ratio: 1.0958382159768851x
 | noise: 2.60%
```
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
autosubmit Merge PR when tree becomes green via auto submit App platform-web Code specifically for the web engine will affect goldens
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants