A peek on WebM in MediaSource Extensions

Alicia Boya (Igalia)

FOMS 2018

Things to like about WebM

It's an open container format.

We can ship it (enabled by default) in Linux distros!

So that people can watch (at least some) YouTube without having to install stuff.

(and that makes some people really happy, I promise)

It allows us to use cool new(ish) open codecs in MSE

VP9 (video)

• Better quality/size ratio that h.264.
• Nowadays with hw support!
• The benefits of inter frame prediction without the worries:
  As far as the user is concerned, PTS = DTS thanks to super-frames.
• If you want the best quality in YouTube playback, you need VP9 (the difference is especially notable in video games, also necessary for 2k/4k).

Opus (audio)

• The codec to end all codecs.
• Best compression than all other codecs in almost all circumstances.
• Lowest codec delay: 26.5ms by default, configurable up to 5ms.
• Handles voice, music and sound effects just fine.
  • No need for crappy voice codecs anymore!

• VP8 compresses less, but is still OK when encoding resources are scarce.
• Vorbis is still cool.
• AV1: WebM is the preferred container format.

Other formats

Stuff you don't have to worry

• PTS vs DTS
• Negative timestamps
• Edit lists
• Pointers (want low latency streaming?)

What is WebM anyway?

It's actually Matroska... when used with open formats.

https://www.matroska.org/technical/specs/index.html

Not everything is rosy

Because everything is broken in multimedia

WebM pain points

• Timescale is usually a power of ten
• Timescale is usually milliseconds
• Durations are optional
• Timestamps are unrestricted
• Buggy muxers make timestamps slightly worse

... the MSE spec expects accurate timestamps and known durations.

What to do with durations?

• Read them when they are present (lucky!)
  • Beware! DefaultDuration is in a different timescale.
• Duration not present, but it's an audio frame: you can parse the audio frame (not ideal, but it is reliable).
• Everything else fails: cry, then...
  • Option a) Use a small default, ensure gaps are not a problem.
  • Option b) Guess. Cry even more when you guess wrong.
  • Option c) Be fancy, wait for a consecutive frame. Cry when that frame never comes.

Timestamps

• Ensure small gaps don't stall the player.
• Overlaps will happen. Assume timestamps are fuzzy.
  • You should probably do that anyway, as there is less than perfect media out there anyway.
    • That applies not only to WebM, by the way!
  • Take the spec with a grain of salt, as everybody is doing.
  • Then maybe fix the spec.

and that's it

you have working WebM in MSE in one more browser.