Add trackpad gesture PointerData types #28571

moffatman · 2021-09-12T15:55:11Z

This commit adds support to the engine core for encoding trackpad gestures for to the framework. This covers zoom, rotate, and pan gestures. The changes to send trackpad gestures from each platform will come in follow-up PRs.

New types of PointerChange: panZoomStart, panZoomUpdate, panZoomEnd
New fields on PointerData to handle two-finger trackpad gesture fields (panX, panY, scale, rotation)
~~Implementation of system APIs to receive trackpad gestures~~
- ~~Chrome OS~~
- ~~iPadOS~~
- ~~Linux~~
- ~~macOS~~
- ~~Windows Win32~~

Not implemented for

Vanilla Android (trackpads are not used)
Web (largely impossible due to browser restrictions)
Windows UWP
Fuchsia

Design Document

Framework PR: flutter/flutter#89944

Addresses flutter/flutter#23604

Pre-launch Checklist

I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
I read the [Tree Hygiene] wiki page, which explains my responsibilities.
I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides].
I listed at least one issue that this PR fixes in the description above.
I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See [testing the engine] for instructions on
writing and running engine tests.
I updated/added relevant documentation (doc comments with ///).
I signed the [CLA].
All existing and new tests are passing.

dkwingsmt

Tried to read as much code as I can, but there are certainly logic that I missed. I found no major problems. Most of the problems are questions related to the design and event model. Also some of the names seem to remain the legacy "gesture" (I might not have pointed out all).

lib/ui/window/pointer_data_packet_converter.cc

lib/ui/window/pointer_data.h

lib/web_ui/lib/src/engine/pointer_converter.dart

lib/web_ui/lib/src/ui/pointer.dart

shell/platform/windows/direct_manipulation.cc

shell/platform/windows/window_win32.cc

shell/platform/windows/window_win32.h

dkwingsmt

All good, except for minor change requests.

lib/ui/pointer.dart

lib/ui/window/pointer_data.h

lib/ui/window/pointer_data_packet_converter.cc

dkwingsmt · 2022-02-03T20:22:20Z

@goderbauer Would you like to take a second review on this?

dkwingsmt

LGTM, still need a second review.

goderbauer

LGTM after doc nits are fixed.

lib/ui/pointer.dart

This reverts commit 2062e57.

alexmarkov · 2022-02-10T16:01:39Z

This change probably caused a new analyzer warning in the Flutter framework:

warning • Missing case clause for 'trackpad' • packages/flutter_test/lib/src/test_pointer.dart:29:5 • missing_enum_constant_in_switch

dkwingsmt · 2022-02-10T16:22:43Z

@alexmarkov It did and it's causing building breakage. Reverting at #31375. I'm wondering why it passed build though.

This reverts commit 2062e57.

aam · 2022-02-10T17:28:29Z

Reverting at #31375. I'm wondering why it passed build though.

It looks like flutter build_and_test_linux_unopt recipe has a problem: it copies engine's sky_engine into flutter's bin/cache/pkg folder, with intent of having flutter framework analyzed together with current engine, but that gets immediately overridden by bin/flutter update-packages. From https://cirrus-ci.com/task/6329915338391552 logs:

cd $FRAMEWORK_PATH/flutter
rm -rf bin/cache/pkg/sky_engine
mkdir -p bin/cache/pkg/
cp -r $ENGINE_PATH/src/out/host_debug_unopt/gen/dart-pkg/sky_engine bin/cache/pkg/
bin/flutter update-packages --local-engine=host_debug_unopt
...
Downloading package sky_engine...                                  114ms
...

cc @godofredoc

* 2a8f388 Roll Skia from a424b619bc40 to ec0af1664478 (6 revisions) (flutter/engine#31352) * ba8f1c5 Add clang-analyzer-* and clang-diagnostic-* to .clang-tidy (flutter/engine#31291) * 1eafb40 Use H5vcc CanvasKit implementation if it is detected. (flutter/engine#31191) * 6d99e90 Roll Dart SDK from 55c93c732da9 to b7eea441d7d1 (4 revisions) (flutter/engine#31353) * 935b46a Roll Fuchsia Mac SDK from 5_CZ81mTD... to 5CmPcHTb1... (flutter/engine#31356) * 244bb25 Roll Skia from ec0af1664478 to 9cb74e90792d (4 revisions) (flutter/engine#31357) * b6b5759 Roll Skia from 9cb74e90792d to 81d4b5d5b45d (6 revisions) (flutter/engine#31360) * 0f6db11 Roll Skia from 81d4b5d5b45d to 1f813e4c7f6d (1 revision) (flutter/engine#31362) * 321a482 Fix AccessibilityBridge crash due to invalid access during ReplaceSemanticsObject (flutter/engine#31351) * 2cc3abb Manual roll of ICU (flutter/engine#31132) * c1e843d Roll Fuchsia Linux SDK from 4VEg4eRJS... to YGS2LvlDy... (flutter/engine#31364) * fe08734 Roll Dart SDK from b7eea441d7d1 to 7e3310bbe1ed (2 revisions) (flutter/engine#31365) * 2062e57 Add trackpad gesture PointerData types (flutter/engine#28571) * 24fe585 Roll Skia from 1f813e4c7f6d to 5a2135af5623 (1 revision) (flutter/engine#31366) * 11bf86a Migrate string encoding conversions to FML (flutter/engine#31334) * 6d9f479 Roll Skia from 5a2135af5623 to 21a92dff8fdc (3 revisions) (flutter/engine#31368) * 417a05e Roll Dart SDK from 7e3310bbe1ed to a9cfcc289ed4 (1 revision) (flutter/engine#31369) * b04ed63 Change link to felt documentation (flutter/engine#31312) * 3e7515a Roll Fuchsia Mac SDK from 5CmPcHTb1... to ZA5ZzabQM... (flutter/engine#31370) * 90effff Revert "Add trackpad gesture PointerData types (#28571)" (flutter/engine#31375) * 3764a8f Change support for VM service message from "The Dart VM Service is listening" to "The Dart VM service is listening" (flutter/engine#31361) * 3d629c5 Fix html gradient rendering (#97762) (flutter/engine#31355) * 963c449 [Android] Show deprecation warnings for Android tests (flutter/engine#31246) * 0be895c Roll Dart SDK from a9cfcc289ed4 to 0041431f5ec7 (1 revision) (flutter/engine#31372) * 18f2faf Roll Skia from 21a92dff8fdc to c5d3326d767d (7 revisions) (flutter/engine#31373) * 1b762d3 Roll Fuchsia Linux SDK from YGS2LvlDy... to yDo1mhBKz... (flutter/engine#31374) * 7b6a7b6 Roll Skia from c5d3326d767d to b6dfd97c5290 (10 revisions) (flutter/engine#31377) * f55b161 [a11y] Delegate UTF8ToUTF16 to FML (flutter/engine#31376) * d36d25f TextEditingDelta Support for the Web (flutter/engine#28527) * aa17186 [fml] Use common FML string encoding utils (flutter/engine#31378) * 97d9ec3 Renamed the scenario tests target to be generic emulator tests. (flutter/engine#28919) * 902717f Roll Skia from b6dfd97c5290 to e1e2a858205f (3 revisions) (flutter/engine#31380) * 877f820 Roll Dart SDK from 0041431f5ec7 to a15d19a0d914 (2 revisions) (flutter/engine#31381) * 2d16729 Roll Skia from e1e2a858205f to 74ce095463e1 (2 revisions) (flutter/engine#31383) * 8d3c0fb [web] PathRef: do not use == with doubles in assertions (flutter/engine#31382) * d1164c1 Move recipes to repository folders. (flutter/engine#31367) * e031e07 Roll Skia from 74ce095463e1 to 82d65d0487bd (1 revision) (flutter/engine#31384) * 5140a44 Define thread priority enum and set thread priority for all threads in Engine (flutter/engine#30605) * 1e46918 Roll Dart SDK from a15d19a0d914 to a3736d4e9b1b (1 revision) (flutter/engine#31397) * 8a28948 Roll Fuchsia Linux SDK from yDo1mhBKz... to UHV3HWM3d... (flutter/engine#31398) * ca86c79 Roll Fuchsia Mac SDK from ZA5ZzabQM... to jjxstNbgZ... (flutter/engine#31399)

This commit adds support to the engine core for encoding trackpad gestures for to the framework. Original PR was #28571

* 2a8f388 Roll Skia from a424b619bc40 to ec0af1664478 (6 revisions) (flutter/engine#31352) * ba8f1c5 Add clang-analyzer-* and clang-diagnostic-* to .clang-tidy (flutter/engine#31291) * 1eafb40 Use H5vcc CanvasKit implementation if it is detected. (flutter/engine#31191) * 6d99e90 Roll Dart SDK from 55c93c732da9 to b7eea441d7d1 (4 revisions) (flutter/engine#31353) * 935b46a Roll Fuchsia Mac SDK from 5_CZ81mTD... to 5CmPcHTb1... (flutter/engine#31356) * 244bb25 Roll Skia from ec0af1664478 to 9cb74e90792d (4 revisions) (flutter/engine#31357) * b6b5759 Roll Skia from 9cb74e90792d to 81d4b5d5b45d (6 revisions) (flutter/engine#31360) * 0f6db11 Roll Skia from 81d4b5d5b45d to 1f813e4c7f6d (1 revision) (flutter/engine#31362) * 321a482 Fix AccessibilityBridge crash due to invalid access during ReplaceSemanticsObject (flutter/engine#31351) * 2cc3abb Manual roll of ICU (flutter/engine#31132) * c1e843d Roll Fuchsia Linux SDK from 4VEg4eRJS... to YGS2LvlDy... (flutter/engine#31364) * fe08734 Roll Dart SDK from b7eea441d7d1 to 7e3310bbe1ed (2 revisions) (flutter/engine#31365) * 2062e57 Add trackpad gesture PointerData types (flutter/engine#28571) * 24fe585 Roll Skia from 1f813e4c7f6d to 5a2135af5623 (1 revision) (flutter/engine#31366) * 11bf86a Migrate string encoding conversions to FML (flutter/engine#31334) * 6d9f479 Roll Skia from 5a2135af5623 to 21a92dff8fdc (3 revisions) (flutter/engine#31368) * 417a05e Roll Dart SDK from 7e3310bbe1ed to a9cfcc289ed4 (1 revision) (flutter/engine#31369) * b04ed63 Change link to felt documentation (flutter/engine#31312) * 3e7515a Roll Fuchsia Mac SDK from 5CmPcHTb1... to ZA5ZzabQM... (flutter/engine#31370) * 90effff Revert "Add trackpad gesture PointerData types (flutter#28571)" (flutter/engine#31375) * 3764a8f Change support for VM service message from "The Dart VM Service is listening" to "The Dart VM service is listening" (flutter/engine#31361) * 3d629c5 Fix html gradient rendering (flutter#97762) (flutter/engine#31355) * 963c449 [Android] Show deprecation warnings for Android tests (flutter/engine#31246) * 0be895c Roll Dart SDK from a9cfcc289ed4 to 0041431f5ec7 (1 revision) (flutter/engine#31372) * 18f2faf Roll Skia from 21a92dff8fdc to c5d3326d767d (7 revisions) (flutter/engine#31373) * 1b762d3 Roll Fuchsia Linux SDK from YGS2LvlDy... to yDo1mhBKz... (flutter/engine#31374) * 7b6a7b6 Roll Skia from c5d3326d767d to b6dfd97c5290 (10 revisions) (flutter/engine#31377) * f55b161 [a11y] Delegate UTF8ToUTF16 to FML (flutter/engine#31376) * d36d25f TextEditingDelta Support for the Web (flutter/engine#28527) * aa17186 [fml] Use common FML string encoding utils (flutter/engine#31378) * 97d9ec3 Renamed the scenario tests target to be generic emulator tests. (flutter/engine#28919) * 902717f Roll Skia from b6dfd97c5290 to e1e2a858205f (3 revisions) (flutter/engine#31380) * 877f820 Roll Dart SDK from 0041431f5ec7 to a15d19a0d914 (2 revisions) (flutter/engine#31381) * 2d16729 Roll Skia from e1e2a858205f to 74ce095463e1 (2 revisions) (flutter/engine#31383) * 8d3c0fb [web] PathRef: do not use == with doubles in assertions (flutter/engine#31382) * d1164c1 Move recipes to repository folders. (flutter/engine#31367) * e031e07 Roll Skia from 74ce095463e1 to 82d65d0487bd (1 revision) (flutter/engine#31384) * 5140a44 Define thread priority enum and set thread priority for all threads in Engine (flutter/engine#30605) * 1e46918 Roll Dart SDK from a15d19a0d914 to a3736d4e9b1b (1 revision) (flutter/engine#31397) * 8a28948 Roll Fuchsia Linux SDK from yDo1mhBKz... to UHV3HWM3d... (flutter/engine#31398) * ca86c79 Roll Fuchsia Mac SDK from ZA5ZzabQM... to jjxstNbgZ... (flutter/engine#31399)

flutter-dashboard bot added embedder Related to the embedder API platform-android platform-ios platform-linux platform-macos platform-web Code specifically for the web engine platform-windows labels Sep 12, 2021

google-cla bot added the cla: yes label Sep 12, 2021

moffatman marked this pull request as ready for review September 13, 2021 06:00

flutter-dashboard bot added the platform-fuchsia label Sep 13, 2021

moffatman force-pushed the trackpad_gestures branch from ae601ab to efabd48 Compare September 19, 2021 20:40

moffatman force-pushed the trackpad_gestures branch from efabd48 to 79fb6f5 Compare September 27, 2021 22:15

dkwingsmt suggested changes Oct 4, 2021

View reviewed changes

moffatman force-pushed the trackpad_gestures branch 2 times, most recently from 68609d4 to ebf828b Compare October 21, 2021 16:56

moffatman mentioned this pull request Oct 21, 2021

Support trackpad gestures in framework flutter/flutter#89944

Merged

8 tasks

moffatman force-pushed the trackpad_gestures branch from cdc6018 to a7fed63 Compare October 28, 2021 05:56

moffatman force-pushed the trackpad_gestures branch 2 times, most recently from 1492cd9 to cc768c2 Compare November 9, 2021 05:40

CaseyHillers changed the base branch from master to main November 15, 2021 18:16

fluttergithubbot force-pushed the master branch from b1a8771 to 2962099 Compare November 15, 2021 18:30

godofredoc changed the base branch from master to main November 24, 2021 07:20

moffatman force-pushed the trackpad_gestures branch 2 times, most recently from 626d3f8 to 03505f6 Compare November 30, 2021 06:18

moffatman force-pushed the trackpad_gestures branch 2 times, most recently from a6d42fc to c6283ab Compare December 9, 2021 00:44

moffatman force-pushed the trackpad_gestures branch from c6283ab to 7f36e7c Compare December 30, 2021 18:09

moffatman force-pushed the trackpad_gestures branch 2 times, most recently from 3bccd0a to 2a659a8 Compare January 14, 2022 23:50

moffatman force-pushed the trackpad_gestures branch from 2a659a8 to 9f2ed78 Compare January 19, 2022 14:32

dkwingsmt suggested changes Feb 3, 2022

View reviewed changes

lib/ui/pointer.dart Outdated Show resolved Hide resolved

lib/ui/window/pointer_data.h Outdated Show resolved Hide resolved

lib/ui/window/pointer_data_packet_converter.cc Outdated Show resolved Hide resolved

moffatman force-pushed the trackpad_gestures branch from d04f9c9 to b1443ee Compare February 4, 2022 01:23

moffatman added 5 commits February 3, 2022 22:59

Implement trackpad gestures in engine

183350a

Remove the platform implementations

66bd4df

Add new device type for trackpad

7b880a8

Add back the necessary changes for Android

262fce5

Address feedback

ed4d2c1

moffatman force-pushed the trackpad_gestures branch from b1443ee to ed4d2c1 Compare February 4, 2022 04:00

dkwingsmt approved these changes Feb 7, 2022

View reviewed changes

goderbauer approved these changes Feb 8, 2022

View reviewed changes

lib/ui/pointer.dart Outdated Show resolved Hide resolved

lib/ui/pointer.dart Outdated Show resolved Hide resolved

Fix doc typos

a165b3a

dkwingsmt merged commit 2062e57 into flutter:main Feb 10, 2022

dkwingsmt added a commit that referenced this pull request Feb 10, 2022

Revert "Add trackpad gesture PointerData types (#28571)"

a9ac1a9

This reverts commit 2062e57.

dkwingsmt mentioned this pull request Feb 10, 2022

Revert "Add trackpad gesture PointerData types" #31375

Merged

dkwingsmt added a commit that referenced this pull request Feb 10, 2022

Revert "Add trackpad gesture PointerData types (#28571)" (#31375)

90effff

This reverts commit 2062e57.

moffatman mentioned this pull request Feb 10, 2022

PointerDeviceKind forwards-compatibility in flutter_test flutter/flutter#98202

Merged

8 tasks

engine-flutter-autoroll mentioned this pull request Feb 11, 2022

Roll Engine from bed591d39448 to 5140a440c0b5 (38 revisions) flutter/flutter#98234

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Feb 11, 2022

2062e57 Add trackpad gesture PointerData types (flutter/engine#28571)

30d47c9

engine-flutter-autoroll mentioned this pull request Feb 11, 2022

Roll Engine from bed591d39448 to 1e46918c13d9 (39 revisions) flutter/flutter#98246

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Feb 11, 2022

2062e57 Add trackpad gesture PointerData types (flutter/engine#28571)

aa9f253

engine-flutter-autoroll mentioned this pull request Feb 11, 2022

Roll Engine from bed591d39448 to ca86c7938d6b (41 revisions) flutter/flutter#98249

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Feb 11, 2022

2062e57 Add trackpad gesture PointerData types (flutter/engine#28571)

65d9563

moffatman mentioned this pull request Feb 11, 2022

Reland: Trackpad gesture PointerData types #31402

Merged

8 tasks

dkwingsmt pushed a commit that referenced this pull request Feb 11, 2022

Reland: Trackpad gesture PointerData types (#31402)

4a3e7a5

This commit adds support to the engine core for encoding trackpad gestures for to the framework. Original PR was #28571

Add trackpad gesture PointerData types #28571

Add trackpad gesture PointerData types #28571

Uh oh!

Conversation

moffatman commented Sep 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

dkwingsmt left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkwingsmt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkwingsmt commented Feb 3, 2022

Uh oh!

dkwingsmt left a comment

Choose a reason for hiding this comment

Uh oh!

goderbauer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexmarkov commented Feb 10, 2022

Uh oh!

dkwingsmt commented Feb 10, 2022

Uh oh!

aam commented Feb 10, 2022

Uh oh!

Uh oh!

moffatman commented Sep 12, 2021 •

edited

Loading

dkwingsmt left a comment •

edited

Loading