[BUG] State of WebKit video codecs

[aslushnikov]

Action Items:

• further investigate webkit video test flakiness
• add a webkit-specific test to make sure MPEG plays and VORBIS doesn't.
• try removing gstreamer1.0-plugins-good from github playwright action
• replace gstreamer1.0-plugins-good with gstreamer1.0-plugins-bad in docker

---

This bug is to track our findings and goals regarding WebKit video support

NOTE Consider reading this great codecs/containers refresher: video and audio on the web

Playwright teams ships Webkit builds on 3 different platforms with an end goal to mimic Safari behavior as close as possible.
In terms of <video> support, this means that we'd like our browsers to satisfy the following conditions:

1. to play video if safari plays it (in other words, support MPEG video)
2. to also not play a video if safari refuses to (in other words, to not play Vorbis videos)

So far we've been focusing on playing videos and didn't spend enough time on limiting video codecs.
Here's the state of the Playwright WebKit browsers across different environments as of July 31, 2020

Windows 10 Desktop

• MPEG: ✅
• VORBIS: ❌

Neither vorbis nor vp8 videos play on WebKit Windows.

Windows Server 2019

• MPEG: ❌
• VORBIS: ❌

We're yet to figure how to install missing video codecs in the server environment.

Mac 10.14/10.14

• MPEG: ✅
• VORBIS: ❌

Codecs are pre-installed in the system - and used by both Safari and Playwright webkit.

Github Actions Ubuntu 18.04 / 20.04

• MPEG: ✅
• VORBIS: ✅

We use microsoft/playwright-github-action to bootstrap virtual environment. We've been fighting video test flakiness that checks that video plays, and as part of the action install the following libraries:

• gstreamer1.0-plugins-good - VP8/VP9, Vorbis
• gstreamer1.0-plugins-bad - MPEG codec
• gstreamer1.0-plugins-base - (added in an attempt to fix video flakiness on CI)

However, it looks like the flakiness has nothing to do with the installed plugins since we still see occasional test failures on CI.
We should try dropping the gstreamer1.0-plugins-good to nuke Vorbis support.

Docker Ubuntu 18.04

• MPEG: ❌
• VORBIS: ✅

We currently only install good plugins that gives us Vorbis and doesn't give MPEG

playwright/docs/docker/Dockerfile.bionic

Lines 32 to 36 in 776f019

	# 3. Install gstreamer and plugins to support video playback in WebKit.
	RUN apt-get update && apt-get install -y --no-install-recommends \
	libgstreamer-gl1.0-0 \
	libgstreamer-plugins-bad1.0-0 \
	gstreamer1.0-plugins-good

[yury-s]

On a related note, we could compile WebKit with --no-gstreamer-gl and eliminate dependency on libgstreamer-gl1.0-0 the savings are negligible though: 37.9 MB vs 38.5 MB (current set of gst packages).

[yury-s]

In order to support h.264 video WebKit requires gstreamer1.0-libav. I tried to get rid of the other gstreamer dependencies since this is the only format we'd like to support but it turns out that libgstreamer-plugins-bad1.0-0 is necessary because of libgstcodecparsers-1.0.so.0 and the test wouldn't work without gstreamer1.0-plugins-good.

[aslushnikov]

So far things seemed to be working fine - there are no plans to act upon this.