Flutter Camera plugin captures video rotated 90 degrees when in Landscape on iPhone

[Dwarfie]

Using the detailed example when I capture video in Landscape on an iPhone the preview is in Portrait aspect ratio with video rotated 90 degrees.
I am uploading the video to Firebase to confirm that video is rotated 90 degrees.

I don't have a physical Android device to check if it is the same.

pubspec.yaml

version: 1.0.0+1

environment: sdk: ">=2.0.0-dev.68.0 <3.0.0"

dependencies: flutter: sdk: flutter

cupertino_icons: ^0.1.2

camera: ^0.4.2

path_provider: ^0.5.0

video_player: ^0.10.0

firebase_core: ^0.2.5

flutter doctor

[✓] Flutter (Channel unknown, v1.1.0, on Mac OS X 10.14.3 18D109, locale en-AU)

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)

[✓] Android Studio (version 3.2)

[✓] Connected device (1 available)

[Dwarfie]

Still no answer on this.
How do I contact the authors of the plugin ?

[quentinleguennec]

Hi @Dwarfie. I've been working on a fix for the rotation on iOS for the last 2 weeks. It's a complicated piece as metadata are handle differently on different platform and different video players. It looks like I found a way to fix it though and I'm creating a few PR. I'll keep you updated.

[quentinleguennec]

I added a fix for video orientation metadata when recording a video with the camera plugin. The PR is this one: flutter/plugins#1452
I also added a fix to the video_player. The PR is here: flutter/plugins#1451
Please don't hesitate to have a look at the PRs and leave comments or suggestions to push the review process.

[Dwarfie]

Hi @quentinleguennec , thanks for the response and the fix. I'm just not sure how I test it.
Is it similar to the way I have referenced map_view in the pubspec.yaml ?

map_view:
git:
url: git://github.com/Eimji/flutter_google_map_view.git

[Dwarfie]

I think I have figured that out, I copied your updated codebase to my project level and added the path: in pubspec.yaml

camera:
path: ./plugins-tengio_camera_added_rotation_metadata_ios/packages/camera/
path_provider: ^0.5.0
video_player:
path: ./plugins-tengio_improved_video_player/packages/video_player/
firebase_storage: ^2.1.0+1

I can report that I can record video in Landscape and have checked it by saving to Firebase Storage

However the view finder on the sample app is messed up for both Portrait and Landscape.

[quentinleguennec]

Hi @Dwarfie
The cleanest way to test and use this modified version of the plugin would be to reference the Github commit in your pubspec.yaml like this:

 video_player:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 9d00507431d3d11e9d7b5f3db91bf4c86614c99b
      path: packages/video_player
  camera:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 0793000a6b4e9f141dd2a18e9336a39cf98b4a07
      path: packages/camera

[quentinleguennec]

Now, when I worked on solving those issue I did 4 fixes in the camera plugin and video_player plugin. And it is quite possible it will only work well only with the 4 fixes together. Here is the list of PR (only one of them has been merged so far and I have yet to get any answer from reviewers on the others):

• [camera] Added orientation metadata to iOS recorded videos. plugins#1452
• [Camera] Improved resolution presets and matched quality between Android and iOS plugins#1403
• [video_player] Fixed orientation and position issue for some videos metadata. plugins#1451
• [video_player] Android: Added missing event.put("event", "completed"); plugins#1470

I created a branch with those 4 fixes merged together, I suggest you try this branch first and see if it works on your side:
https://github.com/Tengio/plugins/tree/tested4you_video_player_camera_merge

You can use it by adding this in your pubspec.yaml:

 video_player:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 310a8f06c4981121a5553b0f6f3a8cdf475b8af1
      path: packages/video_player
  camera:
    git:
      url: git://github.com/tengio/plugins.git
      ref: 310a8f06c4981121a5553b0f6f3a8cdf475b8af1
      path: packages/camera

[Dwarfie]

@quentinleguennec thanks for the response, I have changed the pubspec.yaml to reference the packages from the latest PR.

This fixes the orientation of the _thumbnailWidget that displays after you have captured video.

Unfortunately the Portrait and Landscape _cameraPreviewWidgets still display the same way as the images attached to my previous post.
This is using the example code from here: https://pub.dev/packages/camera#-example-tab-

[quentinleguennec]

@Dwarfie That's because the example doesn't work out of portrait. The reason behind this is that when you rotate the phone flutter rotates everything (and you can't choose what is rotated), and the preview widget ends up rotated. But since only the preview was rotated and not what you are actually recording (hopefully the actual room doesn't rotate when you rotate your phone ^^) the preview is not showing what you would expect.

[quentinleguennec]

One solution is to lock the app on portrait when you are on the recording page, and if you want to update the UI (button and other) listen to changes in device orientation. For this you need to use a 3rd party plugin, flutter doesn't expose enough of the accelerometer and gyroscope to do this without some native code. I did something that looks good using this plugin:
native_device_orientation
PS: if using this plugin you will want to set useSensor to true.

[quentinleguennec]

My solution was to have the preview in full screen in the background (not rotating) and listen to change in orientation (with native_device_orientation) to rotate the buttons individually when the users rotate the device. This is also what the default Android and iOS camera apps do (that's where I draw inspiration).

[Dwarfie]

Using the current version of the camera plugin the Camera Preview fills the whole screen when the phone is in Portrait, and gets rotated when the phone is rotated.

The PR request version has a squashed down preview in Portrait and rotated 90 degrees when phone is in Landscape, but this does actually capture the video in the right orientation.

These shots are form within my app using the current released plugin, and would do the job if the video acted in the same way as your PR version

I may still try to save the video from the image_picker plugin which use the native camera/video controls ( well on iOS I don't have a physical android device yet )

[quentinleguennec]

@Dwarfie Are you using the code from packages/camera/example to do the preview? Because it doesn't display as expected when I build the example on my device and rotate it. (both on Android and iOS).

Could you share the code of the widget where you have the CameraPreview?

[quentinleguennec]

And here is the code we use (with the version of the plugin I modified and the native_device_orientation plugin) to record videos (in a way similar to the Android camera app):

import 'dart:io';

import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:native_device_orientation/native_device_orientation.dart';
import 'package:video_player/video_player.dart';

class VideoRecordingPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp]);
    return Container(
      color: Colors.black,
      child: VideoRecording(),
    );
  }
}

class VideoRecording extends StatefulWidget {
  @override
  _VideoRecordingState createState() => _VideoRecordingState();
}

class _VideoRecordingState extends BasePresenter<VideoRecording>
    with TickerProviderStateMixin<VideoRecording>, VideoControllerMixin
    implements VideoRecordingPresenterView {
  static const int startValue = 30;

  Stream<NativeDeviceOrientation> orientationChangeListener;
  int rotationQuarterTurns = 0;
  RecordingMode mode;
  String videoFilePath;
  VideoRecordingPresenter _presenter;
  List<CameraDescription> _cameras;
  CameraController _controller;
  int _selectedCamera;
  AnimationController _animationController;
  String _filePath;
  VideoPlayerController _videoController;
  VoidCallback _videoListener;

  _VideoRecordingState() {
    this._presenter = VideoRecordingPresenter(this);
  }

  @override
  void initState() {
    NativeDeviceOrientationCommunicator().orientation().then(
        (orientation) => setMountedState(() => rotationQuarterTurns = convertOrientationToQuarterTurns(orientation)));
    orientationChangeListener = NativeDeviceOrientationCommunicator().onOrientationChanged(useSensor: true)
      ..listen((NativeDeviceOrientation orientation) =>
          setMountedState(() => rotationQuarterTurns = convertOrientationToQuarterTurns(orientation)));

    mode = RecordingMode.ready;
    _selectedCamera = 0;
    _cameras = [];
    _animationController = AnimationController(vsync: this, duration: Duration(seconds: startValue));
    _animationController.addStatusListener((status) {
      if (status == AnimationStatus.completed) _presenter.onRecordingTap(true);
    });
    super.initState();
    _presenter.loadCameras();
  }

  @override
  void dispose() {
    SystemChrome.setPreferredOrientations([
      DeviceOrientation.portraitUp,
      DeviceOrientation.portraitDown,
    ]);
    _animationController?.dispose();
    _presenter?.dispose();
    _videoController?.dispose();
    _controller?.dispose();
    super.dispose();
  }

  int convertOrientationToQuarterTurns(NativeDeviceOrientation orientation) {
    int quarterTurns;
    switch (orientation) {
      case NativeDeviceOrientation.portraitUp:
      case NativeDeviceOrientation.unknown:
        quarterTurns = 0;
        break;
      case NativeDeviceOrientation.landscapeLeft:
        quarterTurns = 1;
        break;
      case NativeDeviceOrientation.portraitDown:
        quarterTurns = 2;
        break;
      case NativeDeviceOrientation.landscapeRight:
        quarterTurns = 3;
        break;
    }
    return quarterTurns;
  }

  @override
  Widget build(BuildContext context) => WillPopScope(
        onWillPop: () async {
          _presenter.onCancelRecording();
          return false;
        },
        child: Stack(children: <Widget>[
          buildMainContent(),
          buildVideoRecordingController(),
        ]),
      );

  Container buildMainContent() =>
      Container(child: _controller != null ? buildCameraOrVideoPreview() : ProgressLoader());

  VideoRecordingController buildVideoRecordingController() => VideoRecordingController(
        mode,
        _cameras,
        StepTween(begin: startValue, end: 0).animate(_animationController),
        Tween<double>(begin: 0.0, end: 1.0).animate(_animationController),
        () => _presenter.onCancelRecording(),
        () => _presenter.onSelectNextCamera(_cameras, _selectedCamera),
        () => _presenter.onRecordingTap(mode == RecordingMode.recording),
        () => _presenter.onContinue(_filePath),
        () => _presenter.onRetry(_filePath),
        _presenter.onPlay,
        rotationQuarterTurns,
      );

  Widget buildCameraOrVideoPreview() => mode == RecordingMode.preview ? buildVideoPreview() : buildCameraPreview();

  @override
  void showCameras(CameraController controller, int selectedIndex, List<CameraDescription> cameras) =>
      setMountedState(() {
        _selectedCamera = selectedIndex;
        _cameras = cameras;
        _controller = controller;
      });

  @override
  void startRecording() => setMountedState(() {
        _animationController.forward();
        mode = RecordingMode.recording;
      });

  @override
  void stopRecording() => setMountedState(() {
        _animationController
          ..stop()
          ..reset();
        mode = RecordingMode.end;
      });

  @override
  void updateFilePath(filePath) => setMountedState(() => _filePath = filePath);

  @override
  void retryRecording() => setMountedState(() => mode = RecordingMode.ready);

  @override
  void startPreview() {
    if (mode != RecordingMode.preview) {
      setMountedState(() => mode = RecordingMode.preview);
    }
    _videoListener = () {
      if (mode == RecordingMode.preview && !_videoController.value.isPlaying) {
        _videoController.removeListener(_videoListener);
        setMountedState(() => mode = RecordingMode.end);
        return;
      }
    };
    _videoController = VideoPlayerController.file(File(_filePath))
      ..setLooping(false)
      ..initialize().then((_) => refreshState()) // refresh the state here to get the correct aspect ratio of the video.
      ..addListener(_videoListener)
      ..play();
  }

  Widget buildCameraPreview() => Center(
        child: CameraPreview(_controller),
      );

  double get videoPreviewAspectRatio {
    if (_videoController.value.size == null) {
      return 1.0;
    }

    double aspectRatio = _videoController.value.aspectRatio;
    bool isDeviceInLandscape = rotationQuarterTurns == 1 || rotationQuarterTurns == 3;

    return isDeviceInLandscape ? 1.0 / aspectRatio : aspectRatio;
  }

  Widget buildVideoPreview() => Center(
        child: Container(
          child: AspectRatio(
            aspectRatio: videoPreviewAspectRatio,
            child: RotatedBox(quarterTurns: rotationQuarterTurns, child: VideoPlayer(_videoController)),
          ),
        ),
      );

  @override
  void showCancelConfirmation() =>
      showGenericAlert("upload_alert_delete_post_title", "upload_alert_delete_post_message", [
        "upload_progress_page_alert_action_cancel",
        "upload_progress_page_alert_action_delete_recording",
      ]).then((value) => _presenter.onCancelAnswered("upload_progress_page_alert_action_cancel" == value));
}

[quentinleguennec]

You won't be able to copy-paste it and run it as is, but you should have all you need to adapt it for your code :)

[Dwarfie]

Thanks again for the suggestions, I will try to integrate them into my code.

Nice Party Duck by the way :)

I remember now that I changed the parts of the camera example that I have used on a suggestion from Kenneth Li, and I am using a RoatedBox for the Camera Preview.

/// Display the preview from the camera (or a message if the preview is not available).
  Widget _cameraPreviewWidget() {
    if (controller == null || !controller.value.isInitialized) {
      return const Text(
        'Tap a camera',
        style: TextStyle(
          color: Colors.white,
          fontSize: 24.0,
          fontWeight: FontWeight.w900,
        ),
      );
    } else {
      return RotatedBox(
        quarterTurns: MediaQuery.of(context).orientation == Orientation.landscape ? 3 : 0,
        child: AspectRatio(
          aspectRatio: controller.value.aspectRatio,
          child: CameraPreview(controller),
        ),
      );
    }
  }

[quentinleguennec]

One important change I made to the camera preview in those PR was to change the aspect ratio. Before the aspect ratio on the camera preview was returningheight/width which is the opposite of what it should be. With my changes it becomes width/height as one would expect.

Having the wrong aspect ratio will lead to a weirdly distorted image (as if it was squashed or stretched). The only way I found to know what aspect ratio to use was by checking what the orientation of the device was and invert the aspect ratio if the device is in landscape.

 double get videoPreviewAspectRatio {
    if (_videoController.value.size == null) {
      return 1.0;
    }

    double aspectRatio = _videoController.value.aspectRatio;
    bool isDeviceInLandscape = rotationQuarterTurns == 1 || rotationQuarterTurns == 3;

    return isDeviceInLandscape ? 1.0 / aspectRatio : aspectRatio;
  }

And Mr Ducky says hi ^^

EDIT: This videoPreviewAspectRatio is when displaying the video after recording, not when preview the camera feed. I suggest trying to invert the aspectRatio: controller.value.aspectRatio, in your code into aspectRatio: 1 / controller.value.aspectRatio, to see if it helps.

[nilobarp]

I used ffmpeg to set proper metadata. It is an extra step and requires me to bundle ffmpeg with the app but for time being this a feasible workaround. See https://stackoverflow.com/a/56105289/1790045 for code sample.

[Dwarfie]

@quentinleguennec aspectRatio: 1 / controller.value.aspectRatio, fixed the camera preview for Portrait, and I can capture and save Landscape video referencing your PR. Any idea when the PR will get released ?

[quentinleguennec]

@Dwarfie Good to know it works properly :)
Do you have any other issue with the PR? (To know if I need to fix anything)
I don't know when the PR will be reviewed. Hopefully now that Google IO is over they will have more time. You can leave a comment on the PRs to tell you'd like it reviewed and merged ASAP to encourage them to look at it.

[Dwarfie]

I give Mr Ducky some credit for the fix

[aleksvujic]

You said that your fix works for videos. Does it work for photos as well?

[ShubhraDeshpande]

@aleksvujic It works, using RotatedBox works even for photos.

[escamoteur]

should be solved by flutter/plugins#1952

[tonypottera24]

I think it is not solved, still needs to rotate manually.
camera: ^0.5.7+3

[javiermedina-c]

Took me three days to discover this bug, I'm currently sending images to a server for face recognition, but always I got an error in the server side, but was for the rotation of the image. Any updates?

[jpiabrantes]

Please re-open this issue. It is not solved.

[Aman2two]

Use this lib to get device orientation and use image lib to change the orientation.

nativeDeviceOrientationCommunicator = NativeDeviceOrientationCommunicator();
nativeDeviceOrientationCommunicator.resume();
nativeDeviceOrientationCommunicator
    .onOrientationChanged(useSensor: true)
    .listen((NativeDeviceOrientation event) {
  nativeDeviceOrientation = event;
});

use nativeDeviceOrientation.index for orientation int value where
1-right tilted,2- down portrait,3-left,4- up portrait

https://pub.dev/packages/native_device_orientation

[yurijdvornyk]

Confirming that the issue still exists.

[cristopher19]

someone found a solution to this issue?

[mgalgs]

should be solved by flutter/plugins#1952

@escamoteur this is still broken. Can you please re-open this issue?

[escamoteur]

This is already followed here #39669

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.