[a11y] Support TalkBack reading by word, character, and paragraph #17626

jonahwilliams · 2020-04-10T02:31:20Z

The logic in talkback does not handle reading from the text portion of an accessibility node when checking if that node could support reading at different granularities. See: https://github.com/google/talkback/blob/9b1d5132b074174d02886faccf731e814d69363c/talkback/src/main/java/GranularityTraversal.java#L207

Fix this by always setting the content description if something is not a text field.

flutter/flutter#13406

Fixes flutter/flutter#13406

goderbauer · 2020-04-10T16:33:00Z

shell/platform/android/io/flutter/view/AccessibilityBridge.java

@@ -756,13 +761,7 @@ public AccessibilityNodeInfo createAccessibilityNodeInfo(int virtualViewId) {
    } else if (hasToggledState) {
      result.setChecked(semanticsNode.hasFlag(Flag.IS_TOGGLED));
      result.setClassName("android.widget.Switch");
-      result.setContentDescription(semanticsNode.getValueLabelHint());
-    } else if (!semanticsNode.hasFlag(Flag.SCOPES_ROUTE)) {


Does this change now make SemanticsNodes that are scoping a route and have a label focusable?

I'll update the condition so we don't apply it to those nodes, thanks for jogging my memory a bit on the context

goderbauer · 2020-04-10T16:33:26Z

/cc @xster Do you want to review this one?

xster · 2020-04-10T18:21:11Z

Thanks for the fix @jonahwilliams.

We should add a test for this. The good news is that writing Java tests is a lot easier now than before. See https://github.com/flutter/engine/blob/master/shell/platform/android/test/README.md#L24. The tests and dependencies all build in ninja and you can use robolectric, mockito and all that good stuff to construction inject etc.

Give it a try. Happy to work with you on it if needed.

jonahwilliams · 2020-04-10T23:52:19Z

@xster I updated run_test to use a configured Java so I can point it at Android Studio's JRE. However now I'm getting:

Exception in thread "main" java.lang.UnsupportedClassVersionError: io/flutter/FlutterTestSuite has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

But this is 1.8.0_212-release-1586-b4-5784211 , so it should work according to the readme?

xster · 2020-04-11T00:17:11Z

Are you on a mac? If you just hard-call a specific java binary but don't set your JAVA_HOME, it's gonna do really weird things. What does your JAVA_HOME and /usr/libexec/java_home -V return?

i.e. I think it's going to be a bit more involved to changing the java instance than just calling a specific binary. If documentation is lacking, adding more docs to let people set JAVA_HOME is probably the least scary option.

xster · 2020-04-11T00:17:59Z

shell/platform/android/test/io/flutter/FlutterTestSuite.java

@@ -29,6 +29,7 @@
 import test.io.flutter.embedding.engine.FlutterShellArgsTest;
 import test.io.flutter.embedding.engine.PluginComponentTest;
 import test.io.flutter.embedding.engine.dart.DartExecutorTest;
+import io.flutter.view.AccessibilityBridgeTest;


I assume the formatter will complain about this

perhaps ... I will format another day :)

xster · 2020-04-11T00:19:26Z

testing/run_tests.py

  EXPECTED_VERSION = '1.8'
  # `java -version` is output to stderr. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4380614
-  version_output = subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT)
+  version_output = subprocess.check_output([java_path, '-version'], stderr=subprocess.STDOUT)


if you really want this, you'll probably need an env map too

for JAVA_HOME

Ahh yeah, with JAVA_HOME set everything is working. Thanks!

shell/platform/android/io/flutter/view/AccessibilityBridge.java

shell/platform/android/test/io/flutter/view/AccessibilityBridgeTest.java

jonahwilliams · 2020-04-11T15:29:47Z

I rolled back the run_test.py changes since that wasn't going to be sufficient. Test files are now formatted correctly.

I added the necessary test code to simulate semantic updates, though I only added tests for the conditions I changed

Co-Authored-By: xster <[email protected]>

jonahwilliams · 2020-04-15T16:57:33Z

Friendly ping @xster

xster

doh, I had this tab opened twice and didn't reply. Sorry for the wait. Awesome tests!

xster · 2020-04-15T19:59:26Z

shell/platform/android/test/io/flutter/view/AccessibilityBridgeTest.java

+  }
+
+  /// The encoding for semantics is described in platform_view_android.cc
+  class TestSemanticsUpdate {


if we want this mapping to be maintained manually, can we add notes to each of accessibilitybridge.updatesemantics, platformviewandroid.updatesemantics and here to remind maintainers?

Something like : "This logic is also used in the test fixture at path/to/test.java"?

Sure. Something like "if you change stuff here, don't forget to update the test file as well"

I added a comment to platform_view_android.cc, since that defines the encoding format. Elsewhere it is only consumed

xster · 2020-04-15T21:46:19Z

LGTM!

…utter#17626)

…raph (flutter/engine#17626)

jonahwilliams added 2 commits April 9, 2020 19:22

[a11y] support Android navigation by word, character, etc

7f5f35d

remove more changes

950f148

googlebot added the cla: yes label Apr 10, 2020

jonahwilliams changed the title ~~Fix navigation~~ [a11y] Support TalkBack reading by word, character, and paragraph Apr 10, 2020

jonahwilliams requested a review from goderbauer April 10, 2020 02:31

auto-assign bot requested a review from GaryQian April 10, 2020 02:33

goderbauer reviewed Apr 10, 2020

View reviewed changes

start working on tests

ac31470

xster reviewed Apr 11, 2020

View reviewed changes

add some unit tests, yayy

ab07a63

jonahwilliams commented Apr 11, 2020

View reviewed changes

shell/platform/android/io/flutter/view/AccessibilityBridge.java Outdated Show resolved Hide resolved

jonahwilliams commented Apr 11, 2020

View reviewed changes

shell/platform/android/test/io/flutter/view/AccessibilityBridgeTest.java Show resolved Hide resolved

jonahwilliams added 4 commits April 11, 2020 08:13

format

d3b0a73

revert run_test changes

8de4366

Add missing license header

5383757

Merge branch 'master' of github.com:flutter/engine into fix_navigation

460a576

jonahwilliams and others added 2 commits April 11, 2020 11:16

Update AccessibilityBridgeTest.java

67c0d09

Update shell/platform/android/io/flutter/view/AccessibilityBridge.java

6e6ec2e

Co-Authored-By: xster <[email protected]>

jonahwilliams requested review from xster and goderbauer and removed request for GaryQian April 13, 2020 19:41

xster reviewed Apr 15, 2020

View reviewed changes

xster approved these changes Apr 15, 2020

View reviewed changes

add comment to embedding

f2596ac

jonahwilliams added 2 commits April 15, 2020 16:19

Merge branch 'master' of github.com:flutter/engine into fix_navigation

9750752

Update platform_view_android.cc

a8caf1b

jonahwilliams merged commit 6e1d7f8 into flutter:master Apr 16, 2020

jonahwilliams deleted the fix_navigation branch April 16, 2020 02:07

goderbauer pushed a commit to goderbauer/engine that referenced this pull request Apr 16, 2020

[a11y] Support TalkBack reading by word, character, and paragraph (fl…

b95bc6f

…utter#17626)

DeMonkeyCoder pushed a commit to DeMonkeyCoder/engine that referenced this pull request Apr 16, 2020

[a11y] Support TalkBack reading by word, character, and paragraph (fl…

2e0948d

…utter#17626)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Apr 16, 2020

6e1d7f8 [a11y] Support TalkBack reading by word, character, and parag…

8457799

…raph (flutter/engine#17626)

engine-flutter-autoroll mentioned this pull request Apr 16, 2020

Roll engine e6a2534b63ac..f4d6ce13dcc4 (32 commits) flutter/flutter#54997

Merged

dnfield mentioned this pull request Nov 30, 2020

Unit tests for Android a11y bridge flutter/flutter#46032

Closed

[a11y] Support TalkBack reading by word, character, and paragraph #17626

[a11y] Support TalkBack reading by word, character, and paragraph #17626

Uh oh!

Conversation

jonahwilliams commented Apr 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

goderbauer commented Apr 10, 2020

Uh oh!

xster commented Apr 10, 2020

Uh oh!

jonahwilliams commented Apr 10, 2020

Uh oh!

xster commented Apr 11, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jonahwilliams commented Apr 11, 2020

Uh oh!

jonahwilliams commented Apr 15, 2020

Uh oh!

xster left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xster commented Apr 15, 2020

Uh oh!

Uh oh!

jonahwilliams commented Apr 10, 2020 •

edited

Loading