Who framed the <video> element:
Making sense of
video on the web
Promo
Feb 2021
@opherv
There is like, a lot of video on the internet
500 million video hours daily
100 million video hours daily
https://99firms.com/blog/facebook-video-statistics/
https://www.nytimes.com/interactive/2020/04/07/technology/coronavirus-internet-use.html
How does web video actually work?
Technology
Politics
I'm @OpherV
Creative Developer
Hi!
<img src="cupcakes.png" /><img src="cupcakes.png"
alt="Me eating cupcakes" />
<video src="video.webm"></video><video src="video.webm"></video>Will this play?
Yes
Unless…
you want it to play on different
browsers and devices
<video>
<!-- chrome, firefox, edge -->
<source src="/videos/video.webm" type='video/webm' />
<!-- safari, iOS, ie(mobile safari) -->
<source src="/videos/video.mp4" type='video/mp4' />
</video>
Will this play?
Technically, Yes
but you also want to provide a video experience in proper quality that doesn’t take forever to load
Broadcast
Web Video
Pixel
Pixel
3 * 8 bits = 24 bits
Red, Green, Blue
2,073,600 pixels * 24 bits = ~6.22 MB for one frame
FullHD = 1920 X 1080 = 2,073,600 pixels
Let's assume a constant rate of 23.976 FPS
One second of video = 6.22 * 23.976 = ~149Mb
Average runtime of a Bridgerton episode is 60 mins
One episode = 60 * 149mb = ~9 Gb
Video playing is a race between
Playback
Download
< = 😊
Playback
Download
> = 😡
Playback
Download
Codec
Coder/Decoder
1994
MPEG-2/H.262
1997
MPEG-2 licensing
2003
AVC (H.264)
2013
HEVC (H.265)
1991
MPEG-1/H.261
https://en.wikipedia.org/wiki/Advanced_Video_Coding
https://blog.chiariglione.org/2018/01/
https://blog.chiariglione.org/2018/01/
Video codec licensing terms
with
If Mozilla wanted to to ship
it would cost...
Five MILLION dollars
They are also
computationally expensive!
Codecs are not just
financially expensive
https://www.idownloadblog.com/2015/10/23/apple-tsmc-info-process/
Who's responsible for providing the codec?
Hardware
OS
Browser
💸💸💸💸💸
1994
MPEG-2/H.262
1997
MPEG-2 licensing
2003
AVC (H.264)
2013
HEVC (H.265)
1991
MPEG-1/H.261
2006
Windows Media Video Streaming
Flash Video (~H.263) / 320 X 240
QuickTime
https://blog.codinghorror.com/did-youtube-cut-the-gordian-knot-of-video-codecs/
<object width="425" height="344">
<param name="movie" value="myMovie.mp4"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed src="https://www.youtube.com/v/9sEI1AUFJKw&hl=en_GB&fs=1&"
type="application/x-shockwave-flash" allowscriptaccess="always"
allowfullscreen="true" width="425" height="344"></embed>
</object>Codec
Coder/Decoder
2013
HEVC (H.265)
1991 - 2003
MPEG-1/H.261
MPEG-2/H.262
AVC (H.264)
MPEG licensing
2006
buys YouTube
2007
<video> tag
lands in Opera
2010
releases VP8
(First open codec!)
2011
YouTube
goes HTML5
2018
AOMedia
releases AV1
releases VP9
2014
Cisco releases
OpenH264
for all browsers
Containers
mov / mp4 / webm / mkv
video streams
audio streams
metadata: codec, bitrate, resolution, subtitles, time...
editing
playback
<video>
<source src="/videos/video1.webm"
type='video/webm;codecs="vp9,vorbis"' />
<source src="/videos/video2.webm"
type='video/webm;codecs="vp8,vorbis"' />
<source src="/videos/video3.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
<source src="/videos/video4.mp4"
type='video/mp4;codecs="video/ogg; codecs="theora, vorbis"' />
</video>Will this play?
Yes*
* If your users are at the office or home where they have a really good internet connection and nobody else is using it, and you have no problem with your server exploding
Video files are
BIG
<video>
<source src="https://cdn.com/video1.webm"
type='video/webm;codecs="vp9, vorbis"' />
<source src="https://cdn.com/video2.webm"
type='video/webm;codecs="vp8, vorbis"' />
<source src="https://cdn.com/video3.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
</video>
Will this play?
Yes..?
But you probably care about people watching the video on their mobile phone, or on a train, or anywhere that’s an underpowered device on a shitty bandwidth
Progressive Download
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
Video Streaming
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
Adaptive Bitrate
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
Streaming Standards
https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API
<video>
<!-- streaming sources -->
<source src="https://cdn.com/video.mpd"
type='application/dash+xml' />
<source src="https://cdn.com/video.m3u8"
type='application/x-mpegURL' />
<!-- progressive download sources -->
<source src="https://cdn.com/video.webm"
type='video/webm;codecs="vp8, vorbis"' />
<source src="https://cdn.com/video.mp4"
type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' />
</video>
Will this play?
Finally, YES
Until it doesn't,
because something always goes wrong with video
Errors
stalled - browser isn't getting data and doesn't know why
abort - media data was aborted
suspend - browser isn't getting data intentionally
error - ???
emptied - media has become empty
Do I really have to handle
all of this stuff? 😮
Probably not 🤷
For ornamental purposes (e.g. video hero image) , use a CDN
https://www.panopto.com/blog/the-way-video-works-online-has-changed/
For hosting, encoding and playing video use a platform
For taking video a few steps forwards
Web video must adapt to the browser, hardware, screen resolution and network conditions and more...
🎥🙅🖼️
but it's so much fun! :D
Thanks!
@opherv
eko.engineering
Who framed the <video> element
By Opher Vishnia
