[camera] Allow logical cameras to use all physical cameras via zoom on android 11+ #6150

lucasoskorep · 2022-07-27T06:12:44Z

Cameras in android versions greater than 11 are strongly pushed by the android team to use logical cameras to wrap multiple physical rear cameras into a single "logical camera."

The Flutter camera plugin has an issue where it calculates zoom incorrectly on these cameras and stops the user from being able to change the zoom appropriately to swap between the physical cameras wrapped by the logical camera exposed via the Android API

This change uses the new APIs implemented in Android 11 to correctly calculate zoom, and allow the camera app to swap between all the physical cameras contained within a single Logical camera in android. Users will still see a single Rear cam, but zooming should swap correctly.

Linked to #6091

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

…amera preference. This preference is essential for supporting Logical cameras on the rear which after android 11 wrap multiple cameras (ultrawide, normal, superzoom, etc) into a single rear facing camera. This updates change how zoom functions as well as how min and max zoom is set to be compliant with those updates as long as the user device is on android 11+.

…amera updates.

camsim99

Thank you for doing this! I see this method you used is preferred for Android 11+ so this is great! I just requested some changes and had some questions.

packages/camera/camera_android/CHANGELOG.md

packages/camera/camera_android/android/build.gradle

.../camera/camera_android/android/src/main/java/io/flutter/plugins/camera/CameraProperties.java

...oid/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomLevelFeature.java

...ra_android/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomUtils.java

camsim99 · 2022-08-05T00:33:45Z

...android/src/test/java/io/flutter/plugins/camera/features/zoomlevel/ZoomLevelFeatureTest.java

@@ -40,7 +40,7 @@ public void before() {
    mockSensorArray = mock(Rect.class);

    mockedStaticCameraZoom
-        .when(() -> ZoomUtils.computeZoom(anyFloat(), any(), anyFloat(), anyFloat()))
+        .when(() -> ZoomUtils.computeZoomRect(anyFloat(), any(), anyFloat(), anyFloat()))
        .thenReturn(mockZoomArea);
  }



Do you think we can add a test here that utilizes Robolectric to make sure the proper CaptureRequest is sent when updateBuilder is called?

So Robolectric has some major issues mocking this call

mock(CaptureRequest.Builder.class);

Seems it can only be mocked using our standard test runner - checking the rest of the project nothing mocks this while using mockito.

I did however create a small method to help mock which version of the SDK_INT is running in our code and was able to write a few more tests using this to set our build SDK to R or Q depending on the test.

Oh gotcha. Okay thanks!

camsim99 · 2022-08-05T00:34:39Z

packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/CameraZoom.java

@@ -1,52 +0,0 @@
-// Copyright 2013 The Flutter Authors. All rights reserved.


I agree that this and its corresponding test should be deleted, since this doesn't seem to be used anywhere. @stuartmorgan can you double check me on this? I'm not sure if you have more context.

Looks like it was replaced with ZoomLevelFeature during a refactor and the dead code just accidentally wasn't cleaned up.

Co-authored-by: Camille Simon <[email protected]>

camsim99

Left a couple more comments, but looks good overall! @stuartmorgan or @bparrishMines pinging you for second review!

...oid/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomLevelFeature.java

...ra_android/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomUtils.java

camsim99 · 2022-08-08T22:00:58Z

...android/src/test/java/io/flutter/plugins/camera/features/zoomlevel/ZoomLevelFeatureTest.java

@@ -40,7 +40,7 @@ public void before() {
    mockSensorArray = mock(Rect.class);

    mockedStaticCameraZoom
-        .when(() -> ZoomUtils.computeZoom(anyFloat(), any(), anyFloat(), anyFloat()))
+        .when(() -> ZoomUtils.computeZoomRect(anyFloat(), any(), anyFloat(), anyFloat()))
        .thenReturn(mockZoomArea);
  }



Oh gotcha. Okay thanks!

packages/camera/camera_android/CHANGELOG.md

…r/plugins/camera/features/zoomlevel/ZoomUtils.java Co-authored-by: Camille Simon <[email protected]>

…icalCameras' into feat/allowCameraZoomToUseAllPhysicalCameras

benkuper · 2022-08-25T14:03:23Z

+1 ! This is a high priority necessity for me and it seems that the PR could go on ?
If not yet, how can I test this before the PR is merged ? Sorry, not a Flutter veteran so trying to understand how to use the modified package to test the functionality.
Thank you !

EDIT :
using the following seems to allow me to test the modified plugin, but not sure where to go from there, is there anything to add to the code or is the zoom detection / lens switch automatic ?

dependency_overrides:
  camera_android:
    git:
      dependency: transitive
      url: https://github.com/lucasoskorep/plugins.git
      ref: feat/allowCameraZoomToUseAllPhysicalCameras
      path: packages/camera/camera_android/

  camera:
    git:
      url: https://github.com/lucasoskorep/plugins.git
      ref: feat/allowCameraZoomToUseAllPhysicalCameras
      path: packages/camera/camera/

lucasoskorep · 2022-08-28T16:27:19Z

+1 ! This is a high priority necessity for me and it seems that the PR could go on ? If not yet, how can I test this before the PR is merged ? Sorry, not a Flutter veteran so trying to understand how to use the modified package to test the functionality. Thank you !

EDIT : using the following seems to allow me to test the modified plugin, but not sure where to go from there, is there anything to add to the code or is the zoom detection / lens switch automatic ?
dependency_overrides:
  camera_android:
    git:
      dependency: transitive
      url: https://github.com/lucasoskorep/plugins.git
      ref: feat/allowCameraZoomToUseAllPhysicalCameras
      path: packages/camera/camera_android/

  camera:
    git:
      url: https://github.com/lucasoskorep/plugins.git
      ref: feat/allowCameraZoomToUseAllPhysicalCameras
      path: packages/camera/camera/ 

This is the fix I have for my application for now. Still pending @stuartmorgan's final approval though so sit tight for getting it from the main package itself!

As for functionality any android phone with a logical camera hiding multiple physical cameras will just return the correct min and max zoom values with the fix so no code changes outside of potentially how you are guarding min and max zoom values :)

stuartmorgan-g

Just a few minor nits, but otherwise looks good! (With the caveat that I'm not very familiar with the camera APIs, so I can't do a in-depth review on the substance of the change.)

.../camera/camera_android/android/src/main/java/io/flutter/plugins/camera/CameraProperties.java

stuartmorgan-g · 2022-09-20T14:20:46Z

packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/CameraZoom.java

@@ -1,52 +0,0 @@
-// Copyright 2013 The Flutter Authors. All rights reserved.


Looks like it was replaced with ZoomLevelFeature during a refactor and the dead code just accidentally wasn't cleaned up.

.../camera/camera_android/android/src/main/java/io/flutter/plugins/camera/CameraProperties.java

...oid/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomLevelFeature.java

…icalCameras' into feat/allowCameraZoomToUseAllPhysicalCameras

lucasoskorep · 2022-09-24T03:53:54Z

@stuartmorgan thanks for reviewing the PR! Got the requested changes added in and updated the version to +3 in pubspec and changelog.md so this PR would be up to date. Let me know if there is anything else you'd like added and I can get fixes set up for it :)

GaryQian · 2023-01-05T19:40:06Z

@stuartmorgan This seems to be ready for another look

stuartmorgan-g

Sorry for the really long delay getting back to this; I lost track of the review request :(

…via zoom on android 11+ (flutter/plugins#6150)

* 6ef1bc1da Roll Flutter from 8c2fdb8 to cc7845e (2 revisions) (flutter/plugins#6983) * 56ab33fd1 [shared_pref]: Bump mockito-inline (flutter/plugins#6976) * e85e0f28f [camera] Allow logical cameras to use all physical cameras via zoom on android 11+ (flutter/plugins#6150) * 11361d010 [camera] Use startVideoCapturing and expose concurrent stream/record (flutter/plugins#6815)

…#118682) * 6ef1bc1da Roll Flutter from 8c2fdb8 to cc7845e (2 revisions) (flutter/plugins#6983) * 56ab33fd1 [shared_pref]: Bump mockito-inline (flutter/plugins#6976) * e85e0f28f [camera] Allow logical cameras to use all physical cameras via zoom on android 11+ (flutter/plugins#6150) * 11361d010 [camera] Use startVideoCapturing and expose concurrent stream/record (flutter/plugins#6815)

…n android 11+ (flutter#6150) * Adding support for the android 11+ Camera2 CONTROL_ZOOM_RATIO_RANGE camera preference. This preference is essential for supporting Logical cameras on the rear which after android 11 wrap multiple cameras (ultrawide, normal, superzoom, etc) into a single rear facing camera. This updates change how zoom functions as well as how min and max zoom is set to be compliant with those updates as long as the user device is on android 11+. * Adding updates to the changelog and pubspec to reflect the zoom and camera updates. * comment cleanup for min and max zoom ratio functions * Pull request fixes * Update packages/camera/camera_android/CHANGELOG.md Co-authored-by: Camille Simon <[email protected]> * Update packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/features/zoomlevel/ZoomUtils.java Co-authored-by: Camille Simon <[email protected]> * Updating comments from PR * Fixing variable name formatting, and comment structures * Fixing variable name formatting, and comment structures * Autogormatter updates Co-authored-by: Camille Simon <[email protected]> Co-authored-by: stuartmorgan <[email protected]>

lucasoskorep added 2 commits July 27, 2022 02:06

Adding updates to the changelog and pubspec to reflect the zoom and c…

60cc465

…amera updates.

lucasoskorep requested a review from camsim99 as a code owner July 27, 2022 06:12

github-actions bot added p: camera platform-android labels Jul 27, 2022

comment cleanup for min and max zoom ratio functions

54ec6b4

lucasoskorep mentioned this pull request Jul 27, 2022

[camera] Return all possible cameras on Android #6091

Closed

11 tasks

camsim99 suggested changes Aug 5, 2022

View reviewed changes

lucasoskorep and others added 2 commits August 5, 2022 00:02

Pull request fixes

af4fb8b

Update packages/camera/camera_android/CHANGELOG.md

90e04fc

Co-authored-by: Camille Simon <[email protected]>

lucasoskorep force-pushed the feat/allowCameraZoomToUseAllPhysicalCameras branch from a167025 to 90e04fc Compare August 5, 2022 04:03

camsim99 approved these changes Aug 8, 2022

View reviewed changes

camsim99 requested a review from stuartmorgan-g August 8, 2022 22:06

lucasoskorep and others added 3 commits August 8, 2022 18:15

Update packages/camera/camera_android/android/src/main/java/io/flutte…

ecfb985

…r/plugins/camera/features/zoomlevel/ZoomUtils.java Co-authored-by: Camille Simon <[email protected]>

Updating comments from PR

7a0cb03

Merge remote-tracking branch 'origin/feat/allowCameraZoomToUseAllPhys…

b0e1ea9

…icalCameras' into feat/allowCameraZoomToUseAllPhysicalCameras

stuartmorgan-g suggested changes Sep 20, 2022

View reviewed changes

lucasoskorep added 3 commits September 23, 2022 23:48

Merge branch 'main' into feat/allowCameraZoomToUseAllPhysicalCameras

85ed31a

Fixing variable name formatting, and comment structures

e828549

Merge remote-tracking branch 'origin/feat/allowCameraZoomToUseAllPhys…

b932713

…icalCameras' into feat/allowCameraZoomToUseAllPhysicalCameras

lucasoskorep added 2 commits September 24, 2022 00:03

Fixing variable name formatting, and comment structures

d1826ce

Autogormatter updates

96b3a3f

lucasoskorep requested a review from stuartmorgan-g October 12, 2022 14:54

stuartmorgan-g approved these changes Jan 17, 2023

View reviewed changes

Merge branch 'main' into feat/allowCameraZoomToUseAllPhysicalCameras

459f49f

stuartmorgan-g added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 17, 2023

auto-submit bot merged commit e85e0f2 into flutter:main Jan 17, 2023

engine-flutter-autoroll mentioned this pull request Jan 18, 2023

Roll Plugins from 4e5cf2d2da27 to 11361d01099d (4 revisions) flutter/flutter#118682

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 18, 2023

e85e0f28f [camera] Allow logical cameras to use all physical cameras …

d8dcb26

…via zoom on android 11+ (flutter/plugins#6150)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[camera] Allow logical cameras to use all physical cameras via zoom on android 11+ #6150

[camera] Allow logical cameras to use all physical cameras via zoom on android 11+ #6150

lucasoskorep commented Jul 27, 2022 •

edited

Loading

camsim99 left a comment

camsim99 Aug 5, 2022

lucasoskorep Aug 5, 2022

camsim99 Aug 8, 2022

camsim99 Aug 5, 2022

stuartmorgan-g Sep 20, 2022

camsim99 left a comment •

edited

Loading

camsim99 Aug 8, 2022

benkuper commented Aug 25, 2022 •

edited

Loading

lucasoskorep commented Aug 28, 2022

stuartmorgan-g left a comment

stuartmorgan-g Sep 20, 2022

lucasoskorep commented Sep 24, 2022

GaryQian commented Jan 5, 2023

stuartmorgan-g left a comment

		@@ -1,52 +0,0 @@
		// Copyright 2013 The Flutter Authors. All rights reserved.

[camera] Allow logical cameras to use all physical cameras via zoom on android 11+ #6150

[camera] Allow logical cameras to use all physical cameras via zoom on android 11+ #6150

Conversation

lucasoskorep commented Jul 27, 2022 • edited Loading

Pre-launch Checklist

camsim99 left a comment

Choose a reason for hiding this comment

camsim99 Aug 5, 2022

Choose a reason for hiding this comment

lucasoskorep Aug 5, 2022

Choose a reason for hiding this comment

camsim99 Aug 8, 2022

Choose a reason for hiding this comment

camsim99 Aug 5, 2022

Choose a reason for hiding this comment

stuartmorgan-g Sep 20, 2022

Choose a reason for hiding this comment

camsim99 left a comment • edited Loading

Choose a reason for hiding this comment

camsim99 Aug 8, 2022

Choose a reason for hiding this comment

benkuper commented Aug 25, 2022 • edited Loading

lucasoskorep commented Aug 28, 2022

stuartmorgan-g left a comment

Choose a reason for hiding this comment

stuartmorgan-g Sep 20, 2022

Choose a reason for hiding this comment

lucasoskorep commented Sep 24, 2022

GaryQian commented Jan 5, 2023

stuartmorgan-g left a comment

Choose a reason for hiding this comment

lucasoskorep commented Jul 27, 2022 •

edited

Loading

camsim99 left a comment •

edited

Loading

benkuper commented Aug 25, 2022 •

edited

Loading