[canvaskit] Further improve overlay optimization by splitting pictures #54878

harryterkelsen · 2024-08-29T21:14:19Z

This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.

Fixes flutter/flutter#149863

On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
I listed at least one issue that this PR fixes in the description above.
I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
I updated/added relevant documentation (doc comments with ///).
I signed the CLA.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

flutter-dashboard · 2024-08-29T23:42:11Z

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

Changes reported for pull request #54878 at sha 3fe35e5

harryterkelsen · 2024-08-29T23:54:22Z

There seems to be a problem with shader masks. Will fix then request review

flutter-dashboard · 2024-09-03T23:30:28Z

Golden file changes are available for triage from new commit, Click here to view.

Changes reported for pull request #54878 at sha 11a9b91

flutter-dashboard · 2024-09-23T21:16:09Z

Golden file changes are available for triage from new commit, Click here to view.

Changes reported for pull request #54878 at sha 76ef3d7

eyebrowsoffire

A few comments and suggestions. They aren't really blockers, so take them or leave them.

lib/web_ui/lib/src/engine/canvaskit/embedded_views.dart

eyebrowsoffire

LGTM!

… pictures" (#55401) Reverts #54878 Crashes on real example app with many platform views

…ng pictures (flutter/engine#54878)

…155629) flutter/engine@7cd3d0b...559f2ff 2024-09-24 [email protected] Revert "[canvaskit] Further improve overlay optimization by splitting pictures" (flutter/engine#55401) 2024-09-24 [email protected] [canvaskit] Further improve overlay optimization by splitting pictures (flutter/engine#54878) 2024-09-24 [email protected] [Impeller] fix OES texture usage. (flutter/engine#55331) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

… pictures" (#55402) This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes flutter/flutter#149863 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of #54878 with a fix for scenes with pictures that are eventually entirely clipped out. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

…splitting pictures" (#55402)" (#55456) Reverts: #55402 Initiated by: chingjun Reason for reverting: caused internal tests to fail. See b/369740500 for more details. Original PR Author: harryterkelsen Reviewed By: {yjbanov} This change reverts the following previous change: This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes flutter/flutter#149863 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of #54878 with a fix for scenes with pictures that are eventually entirely clipped out. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

…lutter#155629) flutter/engine@7cd3d0b...559f2ff 2024-09-24 [email protected] Revert "[canvaskit] Further improve overlay optimization by splitting pictures" (flutter/engine#55401) 2024-09-24 [email protected] [canvaskit] Further improve overlay optimization by splitting pictures (flutter/engine#54878) 2024-09-24 [email protected] [Impeller] fix OES texture usage. (flutter/engine#55331) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

… pictures" (#55464) This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes flutter/flutter#149863 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of #54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

…lutter#155629) flutter/engine@7cd3d0b...559f2ff 2024-09-24 [email protected] Revert "[canvaskit] Further improve overlay optimization by splitting pictures" (flutter/engine#55401) 2024-09-24 [email protected] [canvaskit] Further improve overlay optimization by splitting pictures (flutter/engine#54878) 2024-09-24 [email protected] [Impeller] fix OES texture usage. (flutter/engine#55331) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

… pictures" (#55563) This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes flutter/flutter#149863 Fixes flutter/flutter#155833 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of #54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out. It also fixes a performance issue caused by making too many Canvases just to record the size of the picture elements in the scene. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

… pictures" (flutter#55563) This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes flutter/flutter#149863 Fixes flutter/flutter#155833 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of flutter#54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out. It also fixes a performance issue caused by making too many Canvases just to record the size of the picture elements in the scene. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

… pictures" (flutter/engine#55563) This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis. Fixes #149863 Fixes #155833 On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps). This is a reland of flutter/engine#54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out. It also fixes a performance issue caused by making too many Canvases just to record the size of the picture elements in the scene. [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

harryterkelsen added 7 commits August 20, 2024 11:03

Add comment for debugOverlayCanvas

235014f

Merge branch 'main' into canvaskit-overlay-split-pictures

6f5b86b

wip

1594d6d

Merge branch 'main' into canvaskit-overlay-split-pictures

3a42f47

wip

10c348e

Merge branch 'main' into canvaskit-overlay-split-pictures

fe4eed6

WIP on overlay optimization

3fe35e5

github-actions bot added the platform-web Code specifically for the web engine label Aug 29, 2024

Merge branch 'main' into canvaskit-overlay-split-pictures

3ff89b2

flutter-dashboard bot added the will affect goldens label Aug 29, 2024

harryterkelsen added 2 commits September 3, 2024 11:02

Merge branch 'main' into canvaskit-overlay-split-pictures

0720e35

Fix picture optimization for shader mask and backdrop filter

11a9b91

harryterkelsen requested a review from yjbanov September 3, 2024 22:59

harryterkelsen added 14 commits September 4, 2024 10:28

Merge branch 'main' into canvaskit-overlay-split-pictures

187074a

Don't combine backdrop filter pictures

3dd0363

Merge branch 'main' into canvaskit-overlay-split-pictures

3c68665

Merge branch 'main' into canvaskit-overlay-split-pictures

76b9446

Merge branch 'main' into canvaskit-overlay-split-pictures

aacfe1a

wip

42fb3d3

Merge branch 'main' into canvaskit-overlay-split-pictures

e6558e3

Merge branch 'main' into canvaskit-overlay-split-pictures

f57aa72

wip

38b3a54

Merge branch 'main' into canvaskit-overlay-split-pictures

3f24cd0

refactor to visitor pattern with 3 passes for rendering

8ce9337

remove debugging print

2bd9f18

fix licenses

816a2f9

fix license and analysis hints

76ef3d7

harryterkelsen requested a review from eyebrowsoffire September 23, 2024 21:36

eyebrowsoffire approved these changes Sep 23, 2024

View reviewed changes

lib/web_ui/lib/src/engine/canvaskit/embedded_views.dart Outdated Show resolved Hide resolved

lib/web_ui/lib/src/engine/canvaskit/embedded_views.dart Outdated Show resolved Hide resolved

lib/web_ui/lib/src/engine/canvaskit/embedded_views.dart Outdated Show resolved Hide resolved

harryterkelsen added 2 commits September 24, 2024 11:15

respond to review comments

8012708

fix errant import

39c81d7

eyebrowsoffire approved these changes Sep 24, 2024

View reviewed changes

harryterkelsen added the autosubmit Merge PR when tree becomes green via auto submit App label Sep 24, 2024

auto-submit bot merged commit 8682e51 into flutter:main Sep 24, 2024
30 checks passed

harryterkelsen mentioned this pull request Sep 24, 2024

Revert "[canvaskit] Further improve overlay optimization by splitting pictures" #55401

Merged

harryterkelsen added a commit that referenced this pull request Sep 24, 2024

Revert "[canvaskit] Further improve overlay optimization by splitting…

559f2ff

… pictures" (#55401) Reverts #54878 Crashes on real example app with many platform views

engine-flutter-autoroll mentioned this pull request Sep 24, 2024

Roll Flutter Engine from 7cd3d0b1bb2e to 559f2ff31c74 (3 revisions) flutter/flutter#155629

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 24, 2024

8682e519b [canvaskit] Further improve overlay optimization by splitti…

ce84357

…ng pictures (flutter/engine#54878)

yjbanov mentioned this pull request Sep 24, 2024

☂️ Intent to deprecate and remove the HTML renderer in Flutter Web flutter/flutter#145954

Closed

7 tasks

harryterkelsen mentioned this pull request Sep 25, 2024

Reland "[canvaskit] Further improve overlay optimization by splitting pictures" #55402

Merged

8 tasks

auto-submit bot mentioned this pull request Sep 26, 2024

Reverts "Reland "[canvaskit] Further improve overlay optimization by splitting pictures" (#55402)" #55456

Merged

8 tasks

harryterkelsen mentioned this pull request Sep 26, 2024

Reland "[canvaskit] Further improve overlay optimization by splitting pictures" #55464

Merged

8 tasks

harryterkelsen mentioned this pull request Oct 1, 2024

Reland "[canvaskit] Further improve overlay optimization by splitting pictures" #55563

Merged

8 tasks

yjbanov mentioned this pull request Dec 2, 2024

[web] On the web platform, use an <img> tag to show an image if it can't be accessed with CORS flutter/flutter#157755

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[canvaskit] Further improve overlay optimization by splitting pictures #54878

[canvaskit] Further improve overlay optimization by splitting pictures #54878

Uh oh!

harryterkelsen commented Aug 29, 2024 •

edited

Loading

Uh oh!

flutter-dashboard bot commented Aug 29, 2024

Uh oh!

harryterkelsen commented Aug 29, 2024

Uh oh!

flutter-dashboard bot commented Sep 3, 2024

Uh oh!

flutter-dashboard bot commented Sep 23, 2024

Uh oh!

eyebrowsoffire left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eyebrowsoffire left a comment

Uh oh!

Uh oh!

Uh oh!

[canvaskit] Further improve overlay optimization by splitting pictures #54878

[canvaskit] Further improve overlay optimization by splitting pictures #54878

Uh oh!

Conversation

harryterkelsen commented Aug 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

flutter-dashboard bot commented Aug 29, 2024

Uh oh!

harryterkelsen commented Aug 29, 2024

Uh oh!

flutter-dashboard bot commented Sep 3, 2024

Uh oh!

flutter-dashboard bot commented Sep 23, 2024

Uh oh!

eyebrowsoffire left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eyebrowsoffire left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

harryterkelsen commented Aug 29, 2024 •

edited

Loading