Skip to content
This repository was archived by the owner on Feb 22, 2023. It is now read-only.

Commit 9c312d4

Browse files
BradenBagbyBradenBagbyWavvcamsim99bparrishMines
authored
[camera] flip/change camera while recording (split out PR for cam_avfoundation and cam_android) (#7109)
* setDescription in Camera platform interface * example app setup to change description mid recording * AVFoundationCamera method call to setDescription * FLTCam setup to setDescription * captureSession split into video and audio so we will be able to switch cameras without breaking the audio * renamed setDescription to setDescriptionWhileRecording since it can only be used while recording * integration tests fixed * set description while recording integration test * throws error if device not recording and setDescriptionWhileRecording is called * set description while recording test * example project setup * camera preview can be changed while recording * camera switches and keeps surface pointed to mediarecorder * small change to set autofocus when switching while recording * android video record goes through VideoRenderer to apply matrix after switching camera * switch camera uses VideoRenderer * dont use video renderer until user switches camera while recording * rotate based on initial recording direction * VideoRenderer cleanup * flutter results for setDescriptionWhileRecording * error if you setDescriptionWhileRecording while device is not recording * android tests * integration tests * method channel test * main package tests * setDescriptionWhileRecording called while no video was recording test * integration tests * dependency overrides * update readme and version * removed old TODO * removed accidental dev team ID commit * renamed local variables * use captureSessionQueue * fixed local variable name * setupCaptureVideoOutput function * createConnectionWithInput * simplified configureConnection function to re-use code on switching camera * formatting * example project dependency overrides * fixed versioning * formatting * fixed some ios native tests * fixed small bug * dont emit initialized when switching camera * ios formatting * dependency overrides for camera/example * android formatting * ios test formatted * android tests formatted * android format that I missed * other android formatting * final formatting with flutter tool * formatted android again * android license in new file * update-excerpts ran * fixed changelog * removed development team * renames configureConnection to createConnection * renames unimplemented error message * renames setDescriptionWhileRecording error to match android and the other errors * fixes formatting * removes override dependencies from camera_web and camera_windows * removes camera_web override dependency in camera package * Update packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/Camera.java Co-authored-by: Camille Simon <43054281+camsim99@users.noreply.github.com> * Update packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/Camera.java Co-authored-by: Camille Simon <43054281+camsim99@users.noreply.github.com> * Update packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/Camera.java Co-authored-by: Camille Simon <43054281+camsim99@users.noreply.github.com> * Update packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/Camera.java Co-authored-by: Camille Simon <43054281+camsim99@users.noreply.github.com> * reformats camera.java * VideoRenderer uses surface texture timestamp instead of current system time * formats VideoRenderer.java * fixes comments in VideoRenderer.java * Update packages/camera/camera_platform_interface/lib/src/platform_interface/camera_platform.dart Co-authored-by: Maurice Parrish <10687576+bparrishMines@users.noreply.github.com> * Update packages/camera/camera/lib/src/camera_controller.dart Co-authored-by: Maurice Parrish <10687576+bparrishMines@users.noreply.github.com> * renames error typo * frees shaders after program linking * handles eglSwapBuffers errors * extension check guards eglPresentationTimeANDROID * cleans openGL resources * reverted timestamp to use uptimeMillis() * Tests for startPreviewWithVideoRendererStream * fixes exception not being caught * tests for correct rotation to be set * fixes versioning * tests method channel setDescriptionWhileRecording * adds forwarding getter on CameraController to its value's description * dummy commit to fix github test's not finding commit hash * adds override description for FakeController in camera tests * fixes versioning for avfoundation and android * fixes versioning * fixes pubspec versions * ios setDescription * setDescription * android setDescription * formatting * revert * nits and reverts * nits * fixes README * fixes other comments * fixes setDescription override in camera_preview_test * set description test * versions * removes changes on platform_interface_changes * points all packages to platform interface version 2.4 * points to the new platform interface * removes everything that isnt under camera_avfoundation and camera_android * removes dependency overrides in examples * removes version change on camera * removes camera changes that were missed * fixes android version --------- Co-authored-by: BradenBagby <braden.bagby@wavv.com> Co-authored-by: Camille Simon <43054281+camsim99@users.noreply.github.com> Co-authored-by: Maurice Parrish <10687576+bparrishMines@users.noreply.github.com>
1 parent f1a3fea commit 9c312d4

File tree

27 files changed

+1014
-125
lines changed

27 files changed

+1014
-125
lines changed

‎packages/camera/camera_android/CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 0.10.5
2+
3+
* Allows camera to be switched while video recording.
4+
15
## 0.10.4
26

37
* Temporarily fixes issue with requested video profiles being null by falling back to deprecated behavior in that case.

‎packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/Camera.java‎

Lines changed: 142 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -115,13 +115,28 @@ class Camera
115115
* Holds all of the camera features/settings and will be used to update the request builder when
116116
* one changes.
117117
*/
118-
private final CameraFeatures cameraFeatures;
118+
private CameraFeatures cameraFeatures;
119+
120+
private String imageFormatGroup;
121+
122+
/**
123+
* Takes an input/output surface and orients the recording correctly. This is needed because
124+
* switching cameras while recording causes the wrong orientation.
125+
*/
126+
private VideoRenderer videoRenderer;
127+
128+
/**
129+
* Whether or not the camera aligns with the initial way the camera was facing if the camera was
130+
* flipped.
131+
*/
132+
private int initialCameraFacing;
119133

120134
private final SurfaceTextureEntry flutterTexture;
135+
private final ResolutionPreset resolutionPreset;
121136
private final boolean enableAudio;
122137
private final Context applicationContext;
123138
private final DartMessenger dartMessenger;
124-
private final CameraProperties cameraProperties;
139+
private CameraProperties cameraProperties;
125140
private final CameraFeatureFactory cameraFeatureFactory;
126141
private final Activity activity;
127142
/** A {@link CameraCaptureSession.CaptureCallback} that handles events related to JPEG capture. */
@@ -211,6 +226,7 @@ public Camera(
211226
this.applicationContext = activity.getApplicationContext();
212227
this.cameraProperties = cameraProperties;
213228
this.cameraFeatureFactory = cameraFeatureFactory;
229+
this.resolutionPreset = resolutionPreset;
214230
this.cameraFeatures =
215231
CameraFeatures.init(
216232
cameraFeatureFactory, cameraProperties, activity, dartMessenger, resolutionPreset);
@@ -251,6 +267,7 @@ private void prepareMediaRecorder(String outputFilePath) throws IOException {
251267
if (mediaRecorder != null) {
252268
mediaRecorder.release();
253269
}
270+
closeRenderer();
254271

255272
final PlatformChannel.DeviceOrientation lockedOrientation =
256273
((SensorOrientationFeature) cameraFeatures.getSensorOrientation())
@@ -279,6 +296,7 @@ private void prepareMediaRecorder(String outputFilePath) throws IOException {
279296

280297
@SuppressLint("MissingPermission")
281298
public void open(String imageFormatGroup) throws CameraAccessException {
299+
this.imageFormatGroup = imageFormatGroup;
282300
final ResolutionFeature resolutionFeature = cameraFeatures.getResolution();
283301

284302
if (!resolutionFeature.checkIsSupported()) {
@@ -323,14 +341,16 @@ public void onOpened(@NonNull CameraDevice device) {
323341
cameraDevice = new DefaultCameraDeviceWrapper(device);
324342
try {
325343
startPreview();
344+
if (!recordingVideo) // only send initialization if we werent already recording and switching cameras
326345
dartMessenger.sendCameraInitializedEvent(
327-
resolutionFeature.getPreviewSize().getWidth(),
328-
resolutionFeature.getPreviewSize().getHeight(),
329-
cameraFeatures.getExposureLock().getValue(),
330-
cameraFeatures.getAutoFocus().getValue(),
331-
cameraFeatures.getExposurePoint().checkIsSupported(),
332-
cameraFeatures.getFocusPoint().checkIsSupported());
333-
} catch (CameraAccessException e) {
346+
resolutionFeature.getPreviewSize().getWidth(),
347+
resolutionFeature.getPreviewSize().getHeight(),
348+
cameraFeatures.getExposureLock().getValue(),
349+
cameraFeatures.getAutoFocus().getValue(),
350+
cameraFeatures.getExposurePoint().checkIsSupported(),
351+
cameraFeatures.getFocusPoint().checkIsSupported());
352+
353+
} catch (CameraAccessException | InterruptedException e) {
334354
dartMessenger.sendCameraErrorEvent(e.getMessage());
335355
close();
336356
}
@@ -340,7 +360,8 @@ public void onOpened(@NonNull CameraDevice device) {
340360
public void onClosed(@NonNull CameraDevice camera) {
341361
Log.i(TAG, "open | onClosed");
342362

343-
// Prevents calls to methods that would otherwise result in IllegalStateException exceptions.
363+
// Prevents calls to methods that would otherwise result in IllegalStateException
364+
// exceptions.
344365
cameraDevice = null;
345366
closeCaptureSession();
346367
dartMessenger.sendCameraClosingEvent();
@@ -756,7 +777,7 @@ public void startVideoRecording(
756777
if (imageStreamChannel != null) {
757778
setStreamHandler(imageStreamChannel);
758779
}
759-
780+
initialCameraFacing = cameraProperties.getLensFacing();
760781
recordingVideo = true;
761782
try {
762783
startCapture(true, imageStreamChannel != null);
@@ -768,6 +789,13 @@ public void startVideoRecording(
768789
}
769790
}
770791

792+
private void closeRenderer() {
793+
if (videoRenderer != null) {
794+
videoRenderer.close();
795+
videoRenderer = null;
796+
}
797+
}
798+
771799
public void stopVideoRecording(@NonNull final Result result) {
772800
if (!recordingVideo) {
773801
result.success(null);
@@ -778,6 +806,7 @@ public void stopVideoRecording(@NonNull final Result result) {
778806
cameraFeatureFactory.createAutoFocusFeature(cameraProperties, false));
779807
recordingVideo = false;
780808
try {
809+
closeRenderer();
781810
captureSession.abortCaptures();
782811
mediaRecorder.stop();
783812
} catch (CameraAccessException | IllegalStateException e) {
@@ -786,7 +815,7 @@ public void stopVideoRecording(@NonNull final Result result) {
786815
mediaRecorder.reset();
787816
try {
788817
startPreview();
789-
} catch (CameraAccessException | IllegalStateException e) {
818+
} catch (CameraAccessException | IllegalStateException | InterruptedException e) {
790819
result.error("videoRecordingFailed", e.getMessage(), null);
791820
return;
792821
}
@@ -1070,13 +1099,51 @@ public void resumePreview() {
10701099
null, (code, message) -> dartMessenger.sendCameraErrorEvent(message));
10711100
}
10721101

1073-
public void startPreview() throws CameraAccessException {
1102+
public void startPreview() throws CameraAccessException, InterruptedException {
1103+
// If recording is already in progress, the camera is being flipped, so send it through the VideoRenderer to keep the correct orientation.
1104+
if (recordingVideo) {
1105+
startPreviewWithVideoRendererStream();
1106+
} else {
1107+
startRegularPreview();
1108+
}
1109+
}
1110+
1111+
private void startRegularPreview() throws CameraAccessException {
10741112
if (pictureImageReader == null || pictureImageReader.getSurface() == null) return;
10751113
Log.i(TAG, "startPreview");
1076-
10771114
createCaptureSession(CameraDevice.TEMPLATE_PREVIEW, pictureImageReader.getSurface());
10781115
}
10791116

1117+
private void startPreviewWithVideoRendererStream()
1118+
throws CameraAccessException, InterruptedException {
1119+
if (videoRenderer == null) return;
1120+
1121+
// get rotation for rendered video
1122+
final PlatformChannel.DeviceOrientation lockedOrientation =
1123+
((SensorOrientationFeature) cameraFeatures.getSensorOrientation())
1124+
.getLockedCaptureOrientation();
1125+
DeviceOrientationManager orientationManager =
1126+
cameraFeatures.getSensorOrientation().getDeviceOrientationManager();
1127+
1128+
int rotation = 0;
1129+
if (orientationManager != null) {
1130+
rotation =
1131+
lockedOrientation == null
1132+
? orientationManager.getVideoOrientation()
1133+
: orientationManager.getVideoOrientation(lockedOrientation);
1134+
}
1135+
1136+
if (cameraProperties.getLensFacing() != initialCameraFacing) {
1137+
1138+
// If the new camera is facing the opposite way than the initial recording,
1139+
// the rotation should be flipped 180 degrees.
1140+
rotation = (rotation + 180) % 360;
1141+
}
1142+
videoRenderer.setRotation(rotation);
1143+
1144+
createCaptureSession(CameraDevice.TEMPLATE_RECORD, videoRenderer.getInputSurface());
1145+
}
1146+
10801147
public void startPreviewWithImageStream(EventChannel imageStreamChannel)
10811148
throws CameraAccessException {
10821149
setStreamHandler(imageStreamChannel);
@@ -1200,17 +1267,7 @@ private void closeCaptureSession() {
12001267
public void close() {
12011268
Log.i(TAG, "close");
12021269

1203-
if (cameraDevice != null) {
1204-
cameraDevice.close();
1205-
cameraDevice = null;
1206-
1207-
// Closing the CameraDevice without closing the CameraCaptureSession is recommended
1208-
// for quickly closing the camera:
1209-
// https://developer.android.com/reference/android/hardware/camera2/CameraCaptureSession#close()
1210-
captureSession = null;
1211-
} else {
1212-
closeCaptureSession();
1213-
}
1270+
stopAndReleaseCamera();
12141271

12151272
if (pictureImageReader != null) {
12161273
pictureImageReader.close();
@@ -1229,6 +1286,66 @@ public void close() {
12291286
stopBackgroundThread();
12301287
}
12311288

1289+
private void stopAndReleaseCamera() {
1290+
if (cameraDevice != null) {
1291+
cameraDevice.close();
1292+
cameraDevice = null;
1293+
1294+
// Closing the CameraDevice without closing the CameraCaptureSession is recommended
1295+
// for quickly closing the camera:
1296+
// https://developer.android.com/reference/android/hardware/camera2/CameraCaptureSession#close()
1297+
captureSession = null;
1298+
} else {
1299+
closeCaptureSession();
1300+
}
1301+
}
1302+
1303+
private void prepareVideoRenderer() {
1304+
if (videoRenderer != null) return;
1305+
final ResolutionFeature resolutionFeature = cameraFeatures.getResolution();
1306+
1307+
// handle videoRenderer errors
1308+
Thread.UncaughtExceptionHandler videoRendererUncaughtExceptionHandler =
1309+
new Thread.UncaughtExceptionHandler() {
1310+
@Override
1311+
public void uncaughtException(Thread thread, Throwable ex) {
1312+
dartMessenger.sendCameraErrorEvent(
1313+
"Failed to process frames after camera was flipped.");
1314+
}
1315+
};
1316+
1317+
videoRenderer =
1318+
new VideoRenderer(
1319+
mediaRecorder.getSurface(),
1320+
resolutionFeature.getCaptureSize().getWidth(),
1321+
resolutionFeature.getCaptureSize().getHeight(),
1322+
videoRendererUncaughtExceptionHandler);
1323+
}
1324+
1325+
public void setDescriptionWhileRecording(
1326+
@NonNull final Result result, CameraProperties properties) {
1327+
1328+
if (!recordingVideo) {
1329+
result.error("setDescriptionWhileRecordingFailed", "Device was not recording", null);
1330+
return;
1331+
}
1332+
1333+
stopAndReleaseCamera();
1334+
prepareVideoRenderer();
1335+
cameraProperties = properties;
1336+
cameraFeatures =
1337+
CameraFeatures.init(
1338+
cameraFeatureFactory, cameraProperties, activity, dartMessenger, resolutionPreset);
1339+
cameraFeatures.setAutoFocus(
1340+
cameraFeatureFactory.createAutoFocusFeature(cameraProperties, true));
1341+
try {
1342+
open(imageFormatGroup);
1343+
} catch (CameraAccessException e) {
1344+
result.error("setDescriptionWhileRecordingFailed", e.getMessage(), null);
1345+
}
1346+
result.success(null);
1347+
}
1348+
12321349
public void dispose() {
12331350
Log.i(TAG, "dispose");
12341351

‎packages/camera/camera_android/android/src/main/java/io/flutter/plugins/camera/MethodCallHandlerImpl.java‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -354,6 +354,18 @@ public void onMethodCall(@NonNull MethodCall call, @NonNull final Result result)
354354
result.success(null);
355355
break;
356356
}
357+
case "setDescriptionWhileRecording":
358+
{
359+
try {
360+
String cameraName = call.argument("cameraName");
361+
CameraProperties cameraProperties =
362+
new CameraPropertiesImpl(cameraName, CameraUtils.getCameraManager(activity));
363+
camera.setDescriptionWhileRecording(result, cameraProperties);
364+
} catch (Exception e) {
365+
handleException(e, result);
366+
}
367+
break;
368+
}
357369
case "dispose":
358370
{
359371
if (camera != null) {

0 commit comments

Comments
 (0)