Who framed the <video> element:
Making sense of
video on the web
MDC
May 2021
@opherv
There is like, a lot of video on the internet
500 million video hours daily
100 million video hours daily
https://99firms.com/blog/facebook-video-statistics/
https://www.nytimes.com/interactive/2020/04/07/technology/coronavirus-internet-use.html
How does web video actually work?
Technology
Politics
I'm @OpherV
Creative Developer
Hi!
<img src="cupcakes.png" /><img src="cupcakes.png"
alt="Me eating cupcakes" />
<video src="video.webm"></video><video src="video.webm"></video>Will this play?
Yes
Unless…
you want it to play on different
browsers and devices
<video>
<!-- chrome, firefox, edge -->
<source src="/videos/video.webm" type='video/webm' />
<!-- safari, iOS, ie(mobile safari) -->
<source src="/videos/video.mp4" type='video/mp4' />
</video>
Will this play?
Technically, Yes
but you also want to provide a video experience in proper quality that doesn’t take forever to load
Broadcast
Web Video
Pixel
Pixel
3 * 8 bits = 24 bits
Red, Green, Blue
2,073,600 pixels * 24 bits = ~6.22 MB for one frame
FullHD = 1920 X 1080 = 2,073,600 pixels
Let's assume a constant rate of 23.976 FPS
One second of video = 6.22 * 23.976 = ~149Mb
Average runtime of a Star Wars movie is 132 mins
One episode = 132 * 149mb = ~20 Gb
Video playing is a race between
Playback
Download
< = 😊
Playback
Download
> = 😡
Playback
Download
Codec
Coder/Decoder
1994
MPEG-2/H.262
1997
MPEG-2 licensing
2003
AVC (H.264)
2013
HEVC (H.265)
1991
MPEG-1/H.261
https://en.wikipedia.org/wiki/Advanced_Video_Coding
https://blog.chiariglione.org/2018/01/
https://blog.chiariglione.org/2018/01/
Video codec licensing terms
with
If Mozilla wanted to to ship
it would cost...
Five MILLION dollars
They are also
computationally expensive!
Codecs are not just
financially expensive
https://www.idownloadblog.com/2015/10/23/apple-tsmc-info-process/
Who's responsible for providing the codec?
Hardware
OS
Browser
💸💸💸💸💸
1994
MPEG-2/H.262
1997
MPEG-2 licensing
2003
AVC (H.264)
2013
HEVC (H.265)
1991
MPEG-1/H.261
2006
Windows Media Video Streaming
Flash Video (~H.263) / 320 X 240
QuickTime
https://blog.codinghorror.com/did-youtube-cut-the-gordian-knot-of-video-codecs/
<object width="425" height="344">
<param name="movie" value="myMovie.mp4"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="https://www.youtube.com/v/9sEI1AUFJKw&hl=en_GB&fs=1&"
type="application/x-shockwave-flash" allowscriptaccess="always"
allowfullscreen="true" width="425" height="344"></embed>
</object>Codec
Coder/Decoder
2013
HEVC (H.265)
1991 - 2003
MPEG-1/H.261
MPEG-2/H.262
AVC (H.264)
MPEG licensing
2006
buys YouTube
2007
<video> tag
lands in Opera
2010
releases VP8
(First open codec!)
2011
YouTube
goes HTML5
2018
AOMedia
releases AV1
releases VP9
2014
Cisco releases
OpenH264
for all browsers
Containers
mov / mp4 / webm / mkv
video streams
audio streams
metadata: codec, bitrate, resolution, subtitles, time...
editing
playback
<video>
<source src="/videos/video1.webm"
type='video/webm;codecs="vp9,vorbis"' />
<source src="/videos/video2.webm"
type='video/webm;codecs="vp8,vorbis"' />
<source src="/videos/video3.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
<source src="/videos/video4.mp4"
type='video/mp4;codecs="video/ogg; codecs="theora, vorbis"' />
</video>Will this play?
Yes*
* If your users are at the office or home where they have a really good internet connection and nobody else is using it, and you have no problem with your server exploding
Video files are
BIG
<video>
<source src="https://cdn.com/video1.webm"
type='video/webm;codecs="vp9, vorbis"' />
<source src="https://cdn.com/video2.webm"
type='video/webm;codecs="vp8, vorbis"' />
<source src="https://cdn.com/video3.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
</video>
Will this play?
In ideal conditions - yes
But you probably care about people watching the video on their mobile phone, or on a train, or anywhere that’s an underpowered device on a shitty bandwidth
Progressive Download
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
Video Streaming
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
The issue with progressive downloads:
Changing bandwidth 📶📈📉
Adaptive Bitrate
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
Streaming Standards
https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API
<video>
<!-- streaming sources -->
<source src="https://cdn.com/video.mpd"
type='application/dash+xml' />
<source src="https://cdn.com/video.m3u8"
type='application/x-mpegURL' />
<!-- progressive download sources -->
<source src="https://cdn.com/video.webm"
type='video/webm;codecs="vp8, vorbis"' />
<source src="https://cdn.com/video.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
</video>
Will this play?
In theory - yes
but it won't
because something always goes wrong with video
Performance issues
Video is not the only thing working in the browser
Oh you wanted to place UI on top of that?
The browser is not the only thing working on the device
https://blogs.nvidia.com/blog/2009/12/16/whats-the-difference-between-a-cpu-and-a-gpu/
Will this play?
Finally, YES
Until it doesn't,
because something always goes wrong with video
Errors
stalled - browser isn't getting data & doesn't know why
abort - media data was aborted
suspend - browser isn't getting data intentionally
error - ???
emptied - media has become empty
Do I really have to handle
all of this stuff? 😮
Probably not 🤷
For ornamental purposes (e.g. video hero image) , use a CDN
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
For hosting, encoding and playing video use a platform
For taking video a few steps forwards
Web video must adapt to the browser, hardware, screen resolution and network conditions and more...
🎥🙅🖼️
Thanks!
@opherv
eko.engineering
a <video> is not as simple as an <img>