Skip to content
This repository was archived by the owner on Feb 22, 2023. It is now read-only.

[video_player] VTT Support #2878

Merged
merged 76 commits into from
Sep 22, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
9788b63
Web VTT Supported Added
Jul 15, 2020
a66f5d7
Fixes
Jul 15, 2020
6481331
VTT with header added
Jul 15, 2020
8f8b095
VTT with header added
Jul 15, 2020
da1c6e4
VTT with header added
Jul 15, 2020
1a21621
VTT with header added
Jul 15, 2020
0d3549e
New tests added
Jul 15, 2020
b018ea9
tests organized by group
Jul 15, 2020
b92f305
Changelog updated
Jul 15, 2020
1573670
Sample vtt renamed
Jul 15, 2020
1e53fc5
Sample vtt renamed
Jul 15, 2020
9034263
Todo Styles commented
Jul 15, 2020
d9647a9
Sample VTT file fixed
Jul 16, 2020
2ebf99f
Sample VTT file fixed
Jul 16, 2020
43abfd6
Sample VTT file fixed
Jul 16, 2020
a44edea
Credentials added
ferrazrx Jul 16, 2020
d92e820
Credentials added
ferrazrx Jul 16, 2020
10e717c
Merge branch 'ferrazrx/web_vtt' of github.com:ferrazrx/plugins into f…
ferrazrx Jul 16, 2020
98e21c1
[image_picker] Add web support to the example app. (#2816)
ferrazrx Jun 5, 2020
e146366
Fix bug in example (#2801)
ferrazrx Jun 6, 2020
c05a6f1
Update Linux desktop Dockerfile for GTK switch (#2826)
ferrazrx Jun 13, 2020
50044cc
[image_picker] fixes for iOS which doesn't present camera/albums with…
ferrazrx Jun 15, 2020
2aacc0f
[path_provider] Updated documentation reflecting changes needed for t…
ferrazrx Jun 16, 2020
928bfb4
[url_launcher] docs: note about encoding URIs (#2172)
ferrazrx Jun 17, 2020
ee9e13a
[In_App_Purchase]queryPastPurchases() shouldn't block transaction upd…
ferrazrx Jun 22, 2020
bf3e958
[e2e] Fix e2e pixel ratio (#2842)
ferrazrx Jun 23, 2020
91bba0f
Update README for plugin list (#2843)
ferrazrx Jun 24, 2020
459b7f8
[connectivity_for_web] Introduce connectivity_for_web package. (#2820)
ferrazrx Jun 30, 2020
b43b1a8
[url_launcher_web] Adds "tel" and "sms" URL support (#2847)
ferrazrx Jun 30, 2020
db547b6
[shared_preferences_linux] Add support for Linux (#2836)
ferrazrx Jun 30, 2020
7864399
[e2e] Use SettableFuture instead of CompletableFuture (#2854)
ferrazrx Jul 1, 2020
7afa2c2
[connectivity] Endorse connectivity_for_web. (#2853)
ferrazrx Jul 1, 2020
a8a5994
[e2e] Bump version to 0.6.0 (#2855)
ferrazrx Jul 1, 2020
0fdc514
[multiple] Improve video playback in image_picker example (#2819)
ferrazrx Jul 1, 2020
1ca27a7
[image_picker] updated VALID_ARCHS to support iPhone simulator (#2761)
ferrazrx Jul 6, 2020
39ba0ae
[url_launcher_linux] Add Linux url_launcher plugin (#2857)
ferrazrx Jul 7, 2020
7391c73
[url_launcher] Endorse url_launcher_linux (#2863)
ferrazrx Jul 7, 2020
00efa43
[shared_preferences_linux] Add iOS stub (#2865)
ferrazrx Jul 8, 2020
8e1dcad
[shared_preferences] Shared preferences linux endorsement (#2864)
ferrazrx Jul 8, 2020
9fdcb05
[connectivity_for_web] Fix JS Interop in release mode. (#2869)
ferrazrx Jul 13, 2020
f0e286b
[google_sign_in] Bridge google_sign_in and googleapis. (#2824)
ferrazrx Jul 14, 2020
1350f8e
Web VTT Supported Added
ferrazrx Jul 15, 2020
c72a068
Fixes
ferrazrx Jul 15, 2020
226b9c6
VTT with header added
ferrazrx Jul 15, 2020
4554771
VTT with header added
ferrazrx Jul 15, 2020
0b7dde4
VTT with header added
ferrazrx Jul 15, 2020
7aa17ba
VTT with header added
ferrazrx Jul 15, 2020
2f686c2
New tests added
ferrazrx Jul 15, 2020
5d09170
tests organized by group
ferrazrx Jul 15, 2020
78098ca
Changelog updated
ferrazrx Jul 15, 2020
46aecd8
Sample vtt renamed
ferrazrx Jul 15, 2020
cacf4dc
Sample vtt renamed
ferrazrx Jul 15, 2020
5524931
Todo Styles commented
ferrazrx Jul 15, 2020
110c115
Sample VTT file fixed
ferrazrx Jul 16, 2020
808b1a0
Sample VTT file fixed
ferrazrx Jul 16, 2020
a041f54
Sample VTT file fixed
ferrazrx Jul 16, 2020
2a16e2e
Credentials added
ferrazrx Jul 16, 2020
acf7417
Merge branch 'ferrazrx/web_vtt' of github.com:ferrazrx/plugins into f…
ferrazrx Jul 16, 2020
de3c973
Comments fixed
ferrazrx Jul 17, 2020
495b37c
Comments fixed
ferrazrx Jul 17, 2020
fc0d049
Version Changed
ferrazrx Jul 17, 2020
9db4ec1
Version Changed
ferrazrx Jul 17, 2020
c5bbba6
Comments fixed
ferrazrx Jul 22, 2020
7bb0d53
Public comments fixed
ferrazrx Aug 20, 2020
d47ea7f
CHANGELOG.md updated
ferrazrx Aug 20, 2020
4c66c38
Merge remote-tracking branch 'upstream/master' into ferrazrx/web_vtt
ferrazrx Aug 20, 2020
db79449
Format fixed
ferrazrx Aug 20, 2020
fa0d6dc
Version updated
ferrazrx Aug 20, 2020
a4fcbf7
merge master, nnbd migration
Sep 13, 2021
61a22a5
add issue links for TODOs
Sep 13, 2021
14468e5
Merge branch 'master' into ferrazrx/web_vtt
Sep 16, 2021
0d68872
review
Sep 21, 2021
c97876c
Merge branch 'ferrazrx/web_vtt' of github.com:ferrazrx/plugins into f…
Sep 21, 2021
6c457a5
Merge branch 'master' into ferrazrx/web_vtt
Sep 21, 2021
5ffff31
review 2
Sep 21, 2021
c8f7c2c
fix license header
Sep 22, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions packages/video_player/video_player/example/assets/sample.vtt
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
WEBVTT

00:00.500 --> 00:01.000
<v Roger Bingham>We are in New York City

00:01.200 --> 00:02.000
<v Roger Bingham>We’re actually at the Lucern Hotel, just down the street

00:16.000 --> 00:18.000
<v Roger Bingham>from the American Museum of Natural History

00:18.000 --> 00:20.000
<v Roger Bingham>And with me is Neil deGrasse Tyson

00:20.000 --> 00:22.000
<v Roger Bingham>Astrophysicist, Director of the Hayden Planetarium

00:22.000 --> 00:24.000
<v Roger Bingham>at the AMNH.

00:24.000 --> 00:26.000
<v Roger Bingham>Thank you for walking down here.

00:27.000 --> 00:30.000
<v Roger Bingham>And I want to do a follow-up on the last conversation we did.

00:30.000 --> 00:31.500 align:right size:50%
<v Roger Bingham>When we e-mailed—

00:30.500 --> 00:32.500 align:left size:50%
<v Neil deGrasse Tyson>Didn’t we talk about enough in that conversation?

00:32.000 --> 00:35.500 align:right size:50%
<v Roger Bingham>No! No no no no; 'cos 'cos obviously 'cos

00:32.500 --> 00:33.500 align:left size:50%
<v Neil deGrasse Tyson><i>Laughs</i>

00:35.500 --> 00:38.000
<v Roger Bingham>You know I’m so excited my glasses are falling off here.
7 changes: 4 additions & 3 deletions packages/video_player/video_player/example/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ dev_dependencies:
flutter:
uses-material-design: true
assets:
- assets/flutter-mark-square-64.png
- assets/Butterfly-209.mp4
- assets/bumble_bee_captions.srt
- assets/flutter-mark-square-64.png
- assets/Butterfly-209.mp4
- assets/bumble_bee_captions.srt
- assets/sample.vtt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
import 'sub_rip.dart';
export 'sub_rip.dart' show SubRipCaptionFile;

import 'web_vtt.dart';
export 'web_vtt.dart' show WebVTTCaptionFile;

/// A structured representation of a parsed closed caption file.
///
/// A closed caption file includes a list of captions, each with a start and end
Expand All @@ -15,6 +18,7 @@ export 'sub_rip.dart' show SubRipCaptionFile;
///
/// See:
/// * [SubRipCaptionFile].
/// * [WebVTTCaptionFile].
abstract class ClosedCaptionFile {
/// The full list of captions from a given file.
///
Expand Down
168 changes: 168 additions & 0 deletions packages/video_player/video_player/lib/src/web_vtt.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
// Copyright 2020 The Chromium Authors. All rights reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be "Copyright 2013 The Flutter Authors." in both new files, which is why format is unhappy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:convert';

import 'closed_caption_file.dart';
import 'package:html/parser.dart';

/// Represents a [ClosedCaptionFile], parsed from the WebVTT file format.
/// See: https://en.wikipedia.org/wiki/WebVTT
class WebVTTCaptionFile extends ClosedCaptionFile {
/// Parses a string into a [ClosedCaptionFile], assuming [fileContents] is in
/// the WebVTT file format.
/// * See: https://en.wikipedia.org/wiki/WebVTT
WebVTTCaptionFile(this.fileContents)
: _captions = _parseCaptionsFromWebVTTString(fileContents);

/// The entire body of the VTT file.
final String fileContents;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this public?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was probably a copy paste from SubRipCaptionFile in sub_rip.dart. Removed fileContents completely. Also added TODO in sub_rip.dart . It's going to be a breaking change to update it in sub_rip.dart and I think it would be better to update it when there's a meaningful breaking change for this plugin.

I'm not sure how to keep track of this tho. We might forgot about this when doing breaking changes in the plugin.


@override
List<Caption> get captions => _captions;

final List<Caption> _captions;
}

List<Caption> _parseCaptionsFromWebVTTString(String file) {
final List<Caption> captions = <Caption>[];
int number = 1;
for (List<String> captionLines in _readWebVTTFile(file)) {
if (captionLines.length < 2) continue;
print(captionLines);

final int captionNumber = number;
final _StartAndEnd startAndEnd =
_StartAndEnd.fromWebVTTString(captionLines[0]);

final String text = captionLines.sublist(1).join('\n');

//TODO: Handle text format
final String textWithoutFormat = _parseHtmlString(text);

final Caption newCaption = Caption(
number: captionNumber,
start: startAndEnd.start,
end: startAndEnd.end,
text: textWithoutFormat,
);

if (newCaption.start != null && newCaption.end != null) {
captions.add(newCaption);
number++;
}
}

return captions;
}

class _StartAndEnd {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a weird name; could we call it _CaptionRange?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another copy paste from sub_rip, will fix it there too

final Duration start;
final Duration end;

_StartAndEnd(this.start, this.end);

// Assumes format from an VTT file.
// For example:
// 00:09.000 --> 00:11.000
static _StartAndEnd fromWebVTTString(String line) {
final RegExp format =
RegExp(_webVTTTimeStamp + _webVTTArrow + _webVTTTimeStamp);

if (!format.hasMatch(line)) {
return _StartAndEnd(null, null);
}

final List<String> times = line.split(_webVTTArrow);

final Duration start = _parseWebVTTTimestamp(times[0]);
final Duration end = _parseWebVTTTimestamp(times[1]);

return _StartAndEnd(start, end);
}
}

String _parseHtmlString(String htmlString) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this _extractTextFromHtml; I would expect parse to return a tree or similar.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

var document = parse(htmlString);
String parsedString = parse(document.body.text).documentElement.text;
return parsedString;
}

// Parses a time stamp in an VTT file into a Duration.
// For example:
//
// _parseWebVTTimestamp('00:01:08.430')
// returns
// Duration(hours: 0, minutes: 1, seconds: 8, milliseconds: 430)
Duration _parseWebVTTTimestamp(String timestampString) {
if (!RegExp(_webVTTTimeStamp).hasMatch(timestampString)) {
return null;
}

final List<String> dotSections = timestampString.split('.');
final List<String> hoursMinutesSeconds = dotSections[0].split(':');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: timeComponents. There's no guarantee that this has all of these.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


int hours = 0;
int minutes = 0;
int seconds = 0;
List<String> styles;

if (hoursMinutesSeconds.length > 2) {
// Timestamp takes the form of [hours]:[minutes]:[seconds].[milliseconds]
hours = int.parse(hoursMinutesSeconds[0]);
minutes = int.parse(hoursMinutesSeconds[1]);
seconds = int.parse(hoursMinutesSeconds[2]);
} else if (int.parse(hoursMinutesSeconds[0]) > 59) {
// Timestamp takes the form of [hours]:[minutes].[milliseconds]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seemed deeply strange to me so I checked the spec, and this isn't a valid timestamp. We shouldn't handle invalid values unless there's substantial real-world evidence that lots of people have written this (which seems unlikely, because even if other parsers allow this it is incapable of expressing any time within the first 59 hours of a video)

// First position is hours as it's over 59.
hours = int.parse(hoursMinutesSeconds[0]);
minutes = int.parse(hoursMinutesSeconds[1]);
} else {
// Timestamp takes the form of [minutes]:[seconds].[milliseconds]
minutes = int.parse(hoursMinutesSeconds[0]);
seconds = int.parse(hoursMinutesSeconds[1]);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the above is removed, this can all condense to:

int hours = 0;
if (hoursMinutesSeconds.length == 3) {
  hours = int.parse(timeComponents.removeAt(0));
}
final int minutes = int.parse(timeComponents.removeAt(0));
final int seconds = int.parse(timeComponents.removeAt(0));

It would be good to add a safety check that the length is either 2 or 3 and returns null first, as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


List<String> milisecondsStyles = dotSections[1].split(" ");
//TODO: Handle styles data on timestamp
if (milisecondsStyles.length > 1) {
styles = milisecondsStyles.sublist(1);
}
int milliseconds = int.parse(milisecondsStyles[0]);

return Duration(
hours: hours,
minutes: minutes,
seconds: seconds,
milliseconds: milliseconds,
);
}

// Reads on VTT file and splits it into Lists of strings where each list is one
// caption.
List<List<String>> _readWebVTTFile(String file) {
final List<String> lines = LineSplitter.split(file).toList();

final List<List<String>> captionStrings = <List<String>>[];
List<String> currentCaption = <String>[];
int lineIndex = 0;
for (final String line in lines) {
final bool isLineBlank = line.trim().isEmpty;
if (!isLineBlank) {
currentCaption.add(line);
}

if (isLineBlank || lineIndex == lines.length - 1) {
captionStrings.add(currentCaption);
currentCaption = <String>[];
}

lineIndex += 1;
}

return captionStrings;
}

const String _webVTTTimeStamp = r'(\d+):(\d{2})(:\d{2})?\.(\d{3})';
const String _webVTTArrow = r' --> ';
3 changes: 2 additions & 1 deletion packages/video_player/video_player/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ dependencies:
# validation, so we set a ^ constraint.
# TODO(amirh): Revisit this (either update this part in the design or the pub tool).
# https://github.com/flutter/flutter/issues/46264
video_player_web: '>=0.1.1 <2.0.0'
video_player_web: ">=0.1.1 <2.0.0"
html: ^0.14.0+3

flutter:
sdk: flutter
Expand Down
119 changes: 119 additions & 0 deletions packages/video_player/video_player/test/web_vtt.test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:flutter_test/flutter_test.dart';
import 'package:video_player/src/closed_caption_file.dart';
import 'package:video_player/video_player.dart';

void main() {
test('Parses VTT file', () {
final WebVTTCaptionFile parsedFile = WebVTTCaptionFile(_validVTT);

expect(parsedFile.captions.length, 13);

final Caption firstCaption = parsedFile.captions.first;
expect(firstCaption.number, 1);
expect(firstCaption.start, Duration(seconds: 9));
expect(firstCaption.end, Duration(seconds: 11, milliseconds: 430));
expect(firstCaption.text, 'We are in New York City');

final Caption secondCaption = parsedFile.captions[1];
expect(secondCaption.number, 2);
expect(
secondCaption.start,
Duration(minutes: 0, seconds: 13, milliseconds: 0),
);
expect(
secondCaption.end,
Duration(minutes: 0, seconds: 16, milliseconds: 0),
);
expect(secondCaption.text,
"We're actually at the Lucern Hotel, just down the street");

//With styles on timestamp
final Caption lastCaption = parsedFile.captions[12];
expect(lastCaption.number, 13);
expect(
lastCaption.start,
Duration(minutes: 0, seconds: 35, milliseconds: 500),
);
expect(
lastCaption.end,
Duration(minutes: 0, seconds: 38, milliseconds: 0),
);
expect(lastCaption.text,
"You know I'm so excited my glasses are falling off here.");
});

test('Parses VTT file with malformed input', () {
final ClosedCaptionFile parsedFile = WebVTTCaptionFile(_malformedVTT);

expect(parsedFile.captions.length, 1);

final Caption firstCaption = parsedFile.captions.single;
expect(firstCaption.number, 1);
expect(firstCaption.start, Duration(seconds: 13));
expect(firstCaption.end, Duration(seconds: 16, milliseconds: 0));
expect(firstCaption.text, 'Valid');
});
}

const String _validVTT = '''
WEBVTT Kind: captions; Language: en

00:09.000 --> 00:11.430
<v Roger Bingham>We are in New York City

00:13.000 --> 00:16.000
<v Roger Bingham>We're actually at the Lucern Hotel, just down the street

00:16.000 --> 00:18.000
<v Roger Bingham>from the American Museum of Natural History

00:18.000 --> 00:20.000
<v Roger Bingham>And with me is Neil deGrasse Tyson

00:20.000 --> 00:22.000
<v Roger Bingham>Astrophysicist, Director of the Hayden Planetarium

00:22.000 --> 00:24.000
<v Roger Bingham>at the AMNH.

00:24.000 --> 00:26.000
<v Roger Bingham>Thank you for walking down here.

00:27.000 --> 00:30.000
<v Roger Bingham>And I want to do a follow-up on the last conversation we did.

00:30.000 --> 00:31.500 align:end size:50%
<v Roger Bingham>When we e-mailed—

00:30.500 --> 00:32.500 align:start size:50%
<v Neil deGrasse Tyson>Didn't we talk about enough in that conversation?

00:32.000 --> 00:35.500 align:end size:50%
<v Roger Bingham>No! No no no no; 'cos 'cos obviously 'cos

00:32.500 --> 00:33.500 align:start size:50%
<v Neil deGrasse Tyson><i>Laughs</i>

00:35.500 --> 00:38.000 align:start size:50%
<v Roger Bingham>You know I'm so excited my glasses are falling off here.

''';

const String _malformedVTT = '''

WEBVTT Kind: captions; Language: en

00:09.000--> 00:11.430
<Test>This one should be ignored because the arrow needs a space.

00:13.000 --> 00:16.000
<Test>Valid

00:16.000 --> 00:8.000
<Test>This one should be ignored because the time is missing a digit.

''';