-
Notifications
You must be signed in to change notification settings - Fork 6k
[Impeller] Maintain separate queues of GLES operations for each thread in the reactor #56573
Conversation
…d in the reactor Threads that add operations to the ReactorGLES assume that those operations will be executed serially. But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa). The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread. This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions. This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned. Fixes flutter/flutter#158535 Fixes flutter/flutter#158388
It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group. |
We have a reactor_gles_unittest setup. I think this could be tested by enqueuing operations from two different threads, reacting on one, and verifying the other threads operation was not invoked |
@@ -381,6 +381,7 @@ ImageDecoderImpeller::UnsafeUploadTextureToPrivate( | |||
FML_DLOG(ERROR) << decode_error; | |||
return std::make_pair(nullptr, decode_error); | |||
} | |||
command_buffer->WaitUntilScheduled(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is a glFlush required here? Can you add a comment explaining why it's required. We aren't doing this elsewhere and it is going to add a performance hit to the decoder on opengles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WaitUntilScheduled
(which wraps glFlush
) is required here. Without this, there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.
Added a comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.
If that's the case, are we sure we don't need a glFinish
then? The glFlush
is just making sure the commands are submitted to the GPU, but glFinish
would make it synchronize on them actually completing. With a glFlush
couldn't the raster thread potentially be using the texture before the data is finished being uploaded? Or maybe the driver is synchronizing on it if it isn't finished yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
glFinish
is expensive - I can see a major increase in latency when scrolling quickly through the Wonderous home screen pages on a Pixel 3. It takes several seconds for the app to catch up and render all pending images if each image decode does a glFinish
.
However, IIUC the spec does not guarantee that a glFlush
on the IO thread will ensure that the texture is safely usable when the raster thread consumes it on all GLES implementations.
Renamed CommandBuffer::WaitUntilScheduled
to CommandBuffer::WaitUntilCompleted
and changed it to call glFinish
.
We should look into having GLES textures use glFenceSync
/glWaitSync
(flutter/flutter#158963)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Notes from the discord channel:
- https://www.khronos.org/webgl/wiki/SharedResouces says glFinish is required
- skia source uses glFinish on opengles2 and glFenceSync on opengles3
mutable Mutex ops_mutex_; | ||
std::vector<Operation> ops_ IPLR_GUARDED_BY(ops_mutex_); | ||
std::map<std::thread::id, std::vector<Operation>> ops_ IPLR_GUARDED_BY( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to instead key this with EGLContext
? It sounded like Chinmay and Jonah had use cases in mind that revolved around passing the context between multiple threads in a worker pool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently EGL contexts are not passed between threads.
ReactorGLES does support calling AddOperation
on a thread that does not have an active EGLContext
. Using the thread ID as a key allows queueing of operations when the EGLContext
is unavailable.
@@ -105,8 +105,10 @@ bool ReactorGLES::RemoveWorker(WorkerID worker) { | |||
} | |||
|
|||
bool ReactorGLES::HasPendingOperations() const { | |||
auto thread_id = std::this_thread::get_id(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note for future archeology and AIs: I was concerned about the overhead of this call, worrying it would require a syscall. I was wrong, Jason showed me that isn't the case. Pthread is keeping that info in user space.
I tested this patch on upstream/main too, this fixes flutter/flutter#158390. I'll add it to the description. |
Added a test - PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything looks good to me, thanks for adding a test. I just have one open question about wether we want a glFlush or glFinish in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks lgtm
I forgot to mention, this is big. Thanks Jason for wading through this. |
This assert would have blocked the following bug: flutter/flutter#158535 This shouldn't be landed until #56573 lands. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [ ] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…159015) Roll Flutter Engine from 619804c0fbb7 to f649330affa8 (47 revisions) flutter/engine@619804c...f649330 2024-11-16 [email protected] Roll Skia from 1ef3b910e064 to 6b0f264bde33 (6 revisions) (flutter/engine#56635) 2024-11-16 [email protected] Revert "Remove `android_jit_release_x86`." (flutter/engine#56634) 2024-11-15 [email protected] Move SetRenderTargetType to EmbedderTestCompositor (flutter/engine#56626) 2024-11-15 [email protected] Roll Dart SDK from b62dadb14d82 to eccb15756020 (2 revisions) (flutter/engine#56630) 2024-11-15 [email protected] [TextInput] Add TextInputType.webSearch (#15762) (flutter/engine#56428) 2024-11-15 [email protected] [web] Send the correct view ID with semantics actions (flutter/engine#56595) 2024-11-15 [email protected] [macos] early return to suppress clang-tidy warning (flutter/engine#56588) 2024-11-15 [email protected] [Impeller] libImpeller: Tinker on the README to cross reference the SDK and the example. (flutter/engine#56623) 2024-11-15 [email protected] [fuchsia] MaybeRunInitialVsyncCallback should only called once (flutter/engine#56429) 2024-11-15 [email protected] Move mac clangd build from x64 to arm (flutter/engine#56567) 2024-11-15 [email protected] Added assert for opengles thread safety (flutter/engine#56585) 2024-11-15 [email protected] [Impeller] enable OpenGL fallback on Android. (flutter/engine#56591) 2024-11-15 [email protected] [Impeller] Maintain separate queues of GLES operations for each thread in the reactor (flutter/engine#56573) 2024-11-15 [email protected] Roll Dart SDK from abe86ca4eb01 to b62dadb14d82 (2 revisions) (flutter/engine#56615) 2024-11-15 [email protected] Roll Skia from b730eb340852 to 1ef3b910e064 (4 revisions) (flutter/engine#56613) 2024-11-15 [email protected] Roll Fuchsia Linux SDK from JkpuAsLzcmYLzf1iX... to UpSQzyXGUhMfedYIh... (flutter/engine#56611) 2024-11-15 [email protected] Roll Dart SDK from bf0eeb9d929c to abe86ca4eb01 (1 revision) (flutter/engine#56610) 2024-11-15 [email protected] Roll Skia from e5fda8472b21 to b730eb340852 (1 revision) (flutter/engine#56608) 2024-11-15 [email protected] Roll Skia from 8ff65da4b8bf to e5fda8472b21 (1 revision) (flutter/engine#56603) 2024-11-15 [email protected] Move TestMetalContext/Surface to testing namespace (flutter/engine#56602) 2024-11-15 [email protected] Roll Dart SDK from 343b9fc6eafe to bf0eeb9d929c (1 revision) (flutter/engine#56604) 2024-11-15 [email protected] [Impeller] Don't enable KHR_debug on Angle. (flutter/engine#56601) 2024-11-15 [email protected] [Impeller] libImpeller: Reset the GL state when transitioning control back to the embedder. (flutter/engine#56597) 2024-11-15 [email protected] Roll Skia from 8946ea477b42 to 8ff65da4b8bf (2 revisions) (flutter/engine#56600) 2024-11-14 [email protected] [Impeller] Put test components in the testing namespace. (flutter/engine#56594) 2024-11-14 [email protected] [Impeller] libImpeller: Add a warning about OpenGL state being trampled. (flutter/engine#56599) 2024-11-14 [email protected] Embedder: Refactor EmbedderConfigBuilder by backend (flutter/engine#56598) 2024-11-14 [email protected] Migrate Metal translation units to Obj-C++ (flutter/engine#56592) 2024-11-14 [email protected] Roll Dart SDK from db4d7d52db48 to 343b9fc6eafe (1 revision) (flutter/engine#56590) 2024-11-14 [email protected] iOS,macOS: Build/test test_metal_surface_unittests (flutter/engine#56586) 2024-11-14 [email protected] Roll Skia from 3222456e63dc to 8946ea477b42 (2 revisions) (flutter/engine#56587) 2024-11-14 [email protected] Roll Skia from 16178bf63de7 to 3222456e63dc (3 revisions) (flutter/engine#56584) 2024-11-14 [email protected] [Android] Fix Slider semantics double tap behaviors (flutter/engine#56452) 2024-11-14 [email protected] Roll Dart SDK from a3b6652100c8 to db4d7d52db48 (1 revision) (flutter/engine#56581) 2024-11-14 [email protected] Move Vulkan context to EmbedderTestContextVulkan (flutter/engine#56571) 2024-11-14 [email protected] Roll Skia from a076435073fc to 16178bf63de7 (1 revision) (flutter/engine#56579) 2024-11-14 [email protected] Roll Skia from 70553189e4e8 to a076435073fc (1 revision) (flutter/engine#56577) 2024-11-14 [email protected] Roll Skia from fa52f2c1ddba to 70553189e4e8 (1 revision) (flutter/engine#56576) 2024-11-14 [email protected] Roll Dart SDK from 3f05b3540cb2 to a3b6652100c8 (3 revisions) (flutter/engine#56575) 2024-11-14 [email protected] Roll Skia from ddbd8d1cb8d8 to fa52f2c1ddba (3 revisions) (flutter/engine#56574) 2024-11-14 [email protected] [Android] choose 24 bit depth buffer. (flutter/engine#56553) 2024-11-14 [email protected] Roll Skia from 3750b8939c7f to ddbd8d1cb8d8 (1 revision) (flutter/engine#56562) 2024-11-14 [email protected] Move FlMouseCursorHandler from FlView to FlEngine (flutter/engine#56026) 2024-11-14 [email protected] [Impeller] force software resize usage for GLES backend. (flutter/engine#56511) 2024-11-14 [email protected] Roll Dart SDK from 66ab1774bfe9 to 3f05b3540cb2 (2 revisions) (flutter/engine#56564) 2024-11-14 [email protected] Embedder: Refactor EmbedderTestContext creation (flutter/engine#56563) ...
depends on #56573 fixes flutter/flutter#158523 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
…d in the reactor (flutter/engine#56573) Threads that add operations to the ReactorGLES assume that those operations will be executed serially. But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa). The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread. This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions. This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned. Fixes flutter#158535 Fixes flutter#158388 Fixes flutter#158390
This assert would have blocked the following bug: flutter#158535 This shouldn't be landed until flutter/engine#56573 lands. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [ ] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
depends on flutter/engine#56573 fixes flutter#158523 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
Threads that add operations to the ReactorGLES assume that those operations will be executed serially.
But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.
This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions.
This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.
Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
Fixes flutter/flutter#158390