Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

[Impeller] Maintain separate queues of GLES operations for each thread in the reactor #56573

Merged
merged 4 commits into from
Nov 15, 2024

Conversation

jason-simmons
Copy link
Member

@jason-simmons jason-simmons commented Nov 14, 2024

Threads that add operations to the ReactorGLES assume that those operations will be executed serially.

But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.

This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions.

This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.

Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
Fixes flutter/flutter#158390

…d in the reactor

Threads that add operations to the ReactorGLES assume that those operations will be executed serially.

But prior to this change, the ReactorGLES added all operations into one queue.  The reactor would then execute those operations on any thread that can react.
This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations.  So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.

This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread.  If the thread can not currently react, then the operations will be queued until the thread enables reactions.

This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller.  This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.

Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

@jonahwilliams
Copy link
Member

We have a reactor_gles_unittest setup. I think this could be tested by enqueuing operations from two different threads, reacting on one, and verifying the other threads operation was not invoked

@@ -381,6 +381,7 @@ ImageDecoderImpeller::UnsafeUploadTextureToPrivate(
FML_DLOG(ERROR) << decode_error;
return std::make_pair(nullptr, decode_error);
}
command_buffer->WaitUntilScheduled();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a glFlush required here? Can you add a comment explaining why it's required. We aren't doing this elsewhere and it is going to add a performance hit to the decoder on opengles.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WaitUntilScheduled (which wraps glFlush) is required here. Without this, there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.

Added a comment.

Copy link
Member

@gaaclarke gaaclarke Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.

If that's the case, are we sure we don't need a glFinish then? The glFlush is just making sure the commands are submitted to the GPU, but glFinish would make it synchronize on them actually completing. With a glFlush couldn't the raster thread potentially be using the texture before the data is finished being uploaded? Or maybe the driver is synchronizing on it if it isn't finished yet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glFinish is expensive - I can see a major increase in latency when scrolling quickly through the Wonderous home screen pages on a Pixel 3. It takes several seconds for the app to catch up and render all pending images if each image decode does a glFinish.

However, IIUC the spec does not guarantee that a glFlush on the IO thread will ensure that the texture is safely usable when the raster thread consumes it on all GLES implementations.

Renamed CommandBuffer::WaitUntilScheduled to CommandBuffer::WaitUntilCompleted and changed it to call glFinish.

We should look into having GLES textures use glFenceSync/glWaitSync (flutter/flutter#158963)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes from the discord channel:

mutable Mutex ops_mutex_;
std::vector<Operation> ops_ IPLR_GUARDED_BY(ops_mutex_);
std::map<std::thread::id, std::vector<Operation>> ops_ IPLR_GUARDED_BY(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to instead key this with EGLContext? It sounded like Chinmay and Jonah had use cases in mind that revolved around passing the context between multiple threads in a worker pool?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently EGL contexts are not passed between threads.

ReactorGLES does support calling AddOperation on a thread that does not have an active EGLContext. Using the thread ID as a key allows queueing of operations when the EGLContext is unavailable.

@@ -105,8 +105,10 @@ bool ReactorGLES::RemoveWorker(WorkerID worker) {
}

bool ReactorGLES::HasPendingOperations() const {
auto thread_id = std::this_thread::get_id();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for future archeology and AIs: I was concerned about the overhead of this call, worrying it would require a syscall. I was wrong, Jason showed me that isn't the case. Pthread is keeping that info in user space.

@gaaclarke
Copy link
Member

I tested this patch on upstream/main too, this fixes flutter/flutter#158390. I'll add it to the description.

@jason-simmons
Copy link
Member Author

Added a test - PTAL

Copy link
Member

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me, thanks for adding a test. I just have one open question about wether we want a glFlush or glFinish in this case.

Copy link
Member

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks lgtm

@gaaclarke
Copy link
Member

I forgot to mention, this is big. Thanks Jason for wading through this.

@jason-simmons jason-simmons added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024
@auto-submit auto-submit bot merged commit 644f85e into flutter:main Nov 15, 2024
34 checks passed
@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024
gaaclarke added a commit that referenced this pull request Nov 15, 2024
This assert would have blocked the following bug:
flutter/flutter#158535

This shouldn't be landed until
#56573 lands.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 16, 2024
github-merge-queue bot pushed a commit to flutter/flutter that referenced this pull request Nov 16, 2024
…159015)

Roll Flutter Engine from 619804c0fbb7 to f649330affa8 (47 revisions)

flutter/engine@619804c...f649330

2024-11-16 [email protected] Roll Skia from 1ef3b910e064 to
6b0f264bde33 (6 revisions) (flutter/engine#56635)
2024-11-16 [email protected] Revert "Remove
`android_jit_release_x86`." (flutter/engine#56634)
2024-11-15 [email protected] Move SetRenderTargetType to
EmbedderTestCompositor (flutter/engine#56626)
2024-11-15 [email protected] Roll Dart SDK from
b62dadb14d82 to eccb15756020 (2 revisions) (flutter/engine#56630)
2024-11-15 [email protected] [TextInput] Add
TextInputType.webSearch (#15762) (flutter/engine#56428)
2024-11-15 [email protected] [web] Send the correct view ID with
semantics actions (flutter/engine#56595)
2024-11-15 [email protected] [macos] early
return to suppress clang-tidy warning (flutter/engine#56588)
2024-11-15 [email protected] [Impeller] libImpeller: Tinker on the
README to cross reference the SDK and the example.
(flutter/engine#56623)
2024-11-15 [email protected] [fuchsia] MaybeRunInitialVsyncCallback
should only called once (flutter/engine#56429)
2024-11-15 [email protected] Move mac clangd build from x64 to arm
(flutter/engine#56567)
2024-11-15 [email protected] Added assert for
opengles thread safety (flutter/engine#56585)
2024-11-15 [email protected] [Impeller] enable OpenGL fallback on
Android. (flutter/engine#56591)
2024-11-15 [email protected] [Impeller] Maintain
separate queues of GLES operations for each thread in the reactor
(flutter/engine#56573)
2024-11-15 [email protected] Roll Dart SDK from
abe86ca4eb01 to b62dadb14d82 (2 revisions) (flutter/engine#56615)
2024-11-15 [email protected] Roll Skia from b730eb340852 to
1ef3b910e064 (4 revisions) (flutter/engine#56613)
2024-11-15 [email protected] Roll Fuchsia Linux SDK from
JkpuAsLzcmYLzf1iX... to UpSQzyXGUhMfedYIh... (flutter/engine#56611)
2024-11-15 [email protected] Roll Dart SDK from
bf0eeb9d929c to abe86ca4eb01 (1 revision) (flutter/engine#56610)
2024-11-15 [email protected] Roll Skia from e5fda8472b21 to
b730eb340852 (1 revision) (flutter/engine#56608)
2024-11-15 [email protected] Roll Skia from 8ff65da4b8bf to
e5fda8472b21 (1 revision) (flutter/engine#56603)
2024-11-15 [email protected] Move TestMetalContext/Surface to testing
namespace (flutter/engine#56602)
2024-11-15 [email protected] Roll Dart SDK from
343b9fc6eafe to bf0eeb9d929c (1 revision) (flutter/engine#56604)
2024-11-15 [email protected] [Impeller] Don't enable KHR_debug on
Angle. (flutter/engine#56601)
2024-11-15 [email protected] [Impeller] libImpeller: Reset the GL
state when transitioning control back to the embedder.
(flutter/engine#56597)
2024-11-15 [email protected] Roll Skia from 8946ea477b42 to
8ff65da4b8bf (2 revisions) (flutter/engine#56600)
2024-11-14 [email protected] [Impeller] Put test components in the
testing namespace. (flutter/engine#56594)
2024-11-14 [email protected] [Impeller] libImpeller: Add a warning
about OpenGL state being trampled. (flutter/engine#56599)
2024-11-14 [email protected] Embedder: Refactor EmbedderConfigBuilder by
backend (flutter/engine#56598)
2024-11-14 [email protected] Migrate Metal translation units to Obj-C++
(flutter/engine#56592)
2024-11-14 [email protected] Roll Dart SDK from
db4d7d52db48 to 343b9fc6eafe (1 revision) (flutter/engine#56590)
2024-11-14 [email protected] iOS,macOS: Build/test
test_metal_surface_unittests (flutter/engine#56586)
2024-11-14 [email protected] Roll Skia from 3222456e63dc to
8946ea477b42 (2 revisions) (flutter/engine#56587)
2024-11-14 [email protected] Roll Skia from 16178bf63de7 to
3222456e63dc (3 revisions) (flutter/engine#56584)
2024-11-14 [email protected] [Android] Fix Slider semantics double
tap behaviors (flutter/engine#56452)
2024-11-14 [email protected] Roll Dart SDK from
a3b6652100c8 to db4d7d52db48 (1 revision) (flutter/engine#56581)
2024-11-14 [email protected] Move Vulkan context to
EmbedderTestContextVulkan (flutter/engine#56571)
2024-11-14 [email protected] Roll Skia from a076435073fc to
16178bf63de7 (1 revision) (flutter/engine#56579)
2024-11-14 [email protected] Roll Skia from 70553189e4e8 to
a076435073fc (1 revision) (flutter/engine#56577)
2024-11-14 [email protected] Roll Skia from fa52f2c1ddba to
70553189e4e8 (1 revision) (flutter/engine#56576)
2024-11-14 [email protected] Roll Dart SDK from
3f05b3540cb2 to a3b6652100c8 (3 revisions) (flutter/engine#56575)
2024-11-14 [email protected] Roll Skia from ddbd8d1cb8d8 to
fa52f2c1ddba (3 revisions) (flutter/engine#56574)
2024-11-14 [email protected] [Android] choose 24 bit depth
buffer. (flutter/engine#56553)
2024-11-14 [email protected] Roll Skia from 3750b8939c7f to
ddbd8d1cb8d8 (1 revision) (flutter/engine#56562)
2024-11-14 [email protected] Move FlMouseCursorHandler from
FlView to FlEngine (flutter/engine#56026)
2024-11-14 [email protected] [Impeller] force software resize
usage for GLES backend. (flutter/engine#56511)
2024-11-14 [email protected] Roll Dart SDK from
66ab1774bfe9 to 3f05b3540cb2 (2 revisions) (flutter/engine#56564)
2024-11-14 [email protected] Embedder: Refactor EmbedderTestContext
creation (flutter/engine#56563)
...
gaaclarke added a commit that referenced this pull request Nov 18, 2024
depends on #56573
fixes flutter/flutter#158523

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
nick9822 pushed a commit to nick9822/flutter that referenced this pull request Dec 18, 2024
…d in the reactor (flutter/engine#56573)

Threads that add operations to the ReactorGLES assume that those operations will be executed serially.

But prior to this change, the ReactorGLES added all operations into one queue.  The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations.  So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.

This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread.  If the thread can not currently react, then the operations will be queued until the thread enables reactions.

This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller.  This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.

Fixes flutter#158535
Fixes flutter#158388
Fixes flutter#158390
nick9822 pushed a commit to nick9822/flutter that referenced this pull request Dec 18, 2024
This assert would have blocked the following bug:
flutter#158535

This shouldn't be landed until
flutter/engine#56573 lands.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
nick9822 pushed a commit to nick9822/flutter that referenced this pull request Dec 18, 2024
depends on flutter/engine#56573
fixes flutter#158523

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
3 participants