Orchestrating video transcoding in Ruby: A story

Michał Matyas

 

@nerdblogpl

https://nerdblog.pl

You can watch the slides live on your phone/laptop

https://tiny.cc/transcoding

I will avoid talking about the business side of the project.

 

Please don't ask, thank you!

Introduction

There was once a project...

Lesson #1

Hindsight 20/20

It's easy to look back and think "this should've been obvious" but you usually lack all the knowledge and experience at the time.

class Video < ActiveRecord::Base

mount_uploader VideoUploader

process_in_background UploadWorker

before_save

before_processing

Video.status = processing

carrierwave-video (ffmpeg)

after_processing_success

after_processing_failure

Video.status = ready

Video.status = failed

CALLBACK ALL THINGS!!!

(please don't)

Lesson #2

Avoid callbacks

Callbacks make your code harder to reason and harder to isolate, slowly turning everything into a tightly coupled co-dependent mess.

Carrierwave versions

include CarrierWave::Video
include ::CarrierWave::Backgrounder::Delay

storage :file

version :mp4 do
  process :encode_video => [:mp4]
end

version :webm do
  process :encode_video => [:webm]
end

Carrierwave versions

support_format :mp4_1080p, {
  resolution: '1920x1080',
  progress: :on_progress_1080p,
  if: :allow_1080p?
}

support_format :mp4_720p, {
  resolution: '1920x1080',
  progress: :on_progress_720p
}

support_format :mp4_480p, {
  resolution: '852x480',
  custom: '-preset ultrafast -g 5', 
  progress: :on_progress_480p
}

Lesson #3

Don't use Carrierwave versions

Or at least don't use them for anything more complex. They are good for other things (probably).

Consider using Shrine instead.

Video

kind: "720p"

Version

kind: "480p"

Version

kind: nil

Version

kind: "720p"
special: true

Version

kind: "480p"
special: true

Version

kind: nil
special: true

Version

class Video < ActiveRecord::Base

mount_uploader VideoUploader

process_in_background UploadWorker

before_save

before_processing

Video.status = processing

carrierwave-video (ffmpeg)

after_processing_success

after_processing_failure

Video.status = ready

Video.status = failed

class Version < ActiveRecord::Base

mount_uploader VersionUploader

process_in_background VersionUploadWorker

before_save

before_processing

Version.status = processing

carrierwave-video (ffmpeg)

after_processing_success

after_processing_failure

Version.status = ready

Version.status = failed

class Version < ActiveRecord::Base

mount_uploader VersionUploader

process_in_background VersionUploadWorker

before_save

before_processing

Version.status = processing

carrierwave-video (ffmpeg)

after_processing_success

after_processing_failure

Version.status = ready

Version.status = failed

class Version < ActiveRecord::Base

mount_uploader VersionUploader

process_in_background VersionUploadWorker

before_save

before_processing

Version.status = processing

carrierwave-video (ffmpeg)

after_processing_success

after_processing_failure

Version.status = ready

Version.status = failed

class Video < ActiveRecord::Base

Video.status = processing

Video.status = ready

Video

original version gets uploaded

ProcessingWorker

checks for existing versions

store in local cache

creates missing ones

starts processing based on the priority
(low quality video - higher priority because it's gonna be ready fastest)

uploads to S3
(in a separate worker once the version is ready)

processing uses streamio-ffmpeg directly to pass custom arguments to ffmpeg based on the video format and allows transcoding from other versions rather than original file (much much faster)

Lesson #4

Simplest solutions are often best solutions

We would have avoided a lot of pain if we didn't try to do things The Rails Way™ but went with the simplest solution instead.

You live and learn!

But why didn't you use...

  • AWS Lambda - 15 minute of max execution time
     
  • Zencoder, Amazon Elastic Transcoder etc. - expensive
     
  • Docker - not that popular (and well supported in production) at the time

Let's talk about MP4

  • FTYP - File identification
     
  • MDAT - media data (streams)
     
  • MOOV - metadata

Tip #1

Copy streams!

When transcoding things, the best approach is to copy streams and match codecs as much as possible because it's way way faster.

MOOV IS MADE OF ATOMS

  • Fixed parameters of the file
     
  • Specific pointers for each frame of data and audio

MOOV IS MADE OF ATOMS

  • TRKH - file times, duration, speed etc.
     
  • MDIA - Media Data
     
  • STCO - start of each video/audio chunk
     
  • STSC - Sample to chunk atom
     
  • STSZ - Sample size atom

Tip #2

Optimize for streaming

ffmpeg has arguments that make video better at streaming.
​The most important:
-movflags +faststart

Tip #3

Bring Your Own Arguments

When using streamio-ffmpeg just ignore all the DSL they have and go for direct passing of as many arguments as you can.

Tip #4

There are no universal solutions

When transcoding videos from network, sometimes it's faster to download-then-process, sometimes to process on the fly. It heavily depends on the codec though.

Tip #5
Use presets

ffmpeg already has presets for most common configuration options but it's a balance between speed and quality. Use fast presets for low resolution videos.

Tip #6
Use profiles

H.264 has profiles (sets of codec features) but not every device will support all of them.

Go with the highest one you can afford.

Tip #7
Always convert to YUV420

There are different pixel formats that define how color information is stored. Use -pix_fmt yuv420 for max compatibility in browsers.

Tip #8
Collect metadata

It is always a good idea to collect metadata from ffprobe for both the incoming video and processed versions to look for clues when things break

Tip #9
Trust but verify

Sometimes file can be cut during transcoding and still technically be "valid", even if incomplete.

 

BUT...

Tip #10
Trust but verify manually

Be super-careful about invalidating the video or audio based on the duration given from ffprobe. It's sometimes approximated without a warning.

That's it, folks!

 

 

@nerdblogpl

https://nerdblog.pl

sources:

  • https://trac.ffmpeg.org/wiki/Encode/H.264
  • https://www.dfrws.org/sites/default/files/session-files/pres-video_authentication_using_file_structure_and_metadata.pdf
  • http://xhelmboyx.tripod.com/formats/mp4-layout.txt
  • https://www.cnwrecovery.com/manual/MP4FileStructure.html

 

Transcoding videos in Ruby: A story

By Michał Matyas

Made with Slides.com

Transcoding videos in Ruby: A story

  • 2,598

More from Michał Matyas