Voctomix
Full-Stack OpenSource HD-Capable Live-Video-Mixer
MaZderMind
& the C3VOC
C3VOC Usual Setup
?
?
Professional Hardware
"StudioTech 110 - Blackmagic ATEM 4K switchers: Part 2 of 3 ATEM Production Studio 4K" by "StudioTech TV" - Creative Commons-Lizenz mit Quellenangabe
Professional Software?
"StudioTech 111 - Blackmagic ATEM 4K switchers: Part 3 of 3 ATEM 1 M/E Production Studio 4K" by "StudioTech TV" - Creative Commons-Lizenz mit Quellenangabe
- OpenSource
- Software
- Commodity Hardware
Replicable
GStreamer
to the rescue!!!
GStreamer
"a pipeline-based multimedia framework"
gst-launch-1.0 \
videotestsrc ! \
video/x-raw,width=1024,height=576 ! \
ximagesink
- Generate Test-Video
- Fixate some Parameters (width & height)
- Let some be auto-negotiated (framerate, format)
- Pipe into X-Server Sink
- Sink does the scheduling
GStreamer
plugin introspection
gst-inspect-1.0 videotestsrc
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV61, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE, NV12_64Z32, A420_10LE, A420_10BE, A422_10LE, A422_10BE, A444_10LE, A444_10BE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-bayer
format: { bggr, rggb, grbg, gbrg }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "videotestsrc0"
pattern : Type of test pattern to generate
flags: readable, writable
Enum "GstVideoTestSrcPattern" Default: 0, "smpte"
(0): smpte - SMPTE 100% color bars
(1): snow - Random (television snow)
(2): black - 100% Black
(3): white - 100% White
…
GStreamer
complex pipeline
gst-launch-1.0 --eos-on-shutdown \
videotestsrc ! \
video/x-raw,width=1024,height=576,framerate=25/1,format=UYVY ! \
videoconvert ! \
x264enc bitrate=4000 ! \
queue ! \
mux. \
\
audiotestsrc ! \
audio/x-raw,format=S16LE,channels=2,rate=48000 ! \
lamemp3enc quality=6 ! \
queue ! \
mux. \
\
mp4mux faststart=false name=mux ! \
filesink location=lala.mp4
GStreamer
compositor
gst-launch-1.0 --eos-on-shutdown \
videotestsrc pattern=smpte ! \
video/x-raw,width=1024,height=576,framerate=25/1,format=UYVY ! \
queue ! \
mix. \
\
videotestsrc pattern=ball ! \
video/x-raw,width=1024,height=576,framerate=25/1,format=UYVY ! \
queue ! \
mix. \
\
compositor name=mix \
sink_1::xpos=50 sink_1::ypos=50 \
sink_1::width=400 sink_1::height=225 \
sink_1::zindex=1 sink_1::alpha=0.75 ! \
videoconvert ! \
ximagesink
Reliability of Sources
- Sources are unreliable. Period.
- Cables can be unplugged
- Power can be shut down
- Requirements can Change
- A GStreamer-Pipeline is a static construct
- Dynamic changes are possible but complicated
Blocking Barriers
- inter*-Elements
- Works like a clutch
- F*cks up timestamps
- Screws up A/V Sync
- WiP rewrite:
http://bit.do/gstlocal - Could use some help
- Get in touch!
Composition
Fullscreen
Picture in Picture
Side by Side Preview
Side by Side Preview
IOs & Flows
IOs & Flows
17000… VSource** (Stream-Blanker) ---\
18000 ASource** (Stream-Blanker) ----\
\
16000 VSource (Background) \
\ \
--> VideoMix \
/ \ -> StreamBlanker** -> StreamOutputPort** 15000
/ \ /
/ ------> OutputPort 11000
/ / \-> Encoder* -> PreviewPort* 12000
/ /
/----- -> AudioMix
/
10000… AVSource --> MirrorPort 13000…
\-> Encoder* -> PreviewPort* 14000…
9999 Control-Server
9998 GstNetTimeProvider Network-Clock
IOs & Flows
- Core only mixes Video
- Sources and Sinks are external Scripts
- Camera-Source
- Slide-Source
- Recording-Sink
- Streaming-Sink
- Lots of Options!
- GUI is External
on same System or Notebook
IOs & Flows
- Raw I420 + Raw S16LE in Matroska
(for Timing and Metadata) - via TCP ~791 MBit/s
loopback or dedicated NIC (or 10GE)
For Streaming or Recording - Preview-Outs: JPEG + Raw S16LE in Matroska
~30-50 MBit/s
For GUI
Control Protocol
Control Protocol
< set_video_a cam1
> video_status cam1 cam2
< set_composite_mode side_by_side_equal
> composite_mode side_by_side_equal
< get_video
> video_status cam1 cam2
< set_stream_blank pause
> stream_status blank pause
… meanwhile in another control-server connection
> video_status cam1 cam2
> composite_mode side_by_side_equal
> stream_status blank pause
Control Protocol
- rlwrap telnet
- nc
- http / html?
- scriptable!
The GUI
…could be better
State and Future Plans
Used by
- C3VOC (excerpt)
- PyConAU
- DebConf17 (partitial)
- linux.conf.au (partitial)
Future Plans
- Stability & Reliability
- Cropping slide sources
- GUI Improvements
- f.lux Compensator
- Performance
- Put stuff onto the GPU
- Documentation
- Improveme 1st Impression
Code
https://github.com/voc/voctomix
https://github.com/voc/voctomix/tree/master/voctocore
https://github.com/voc/voctomix/tree/master/voctogui
https://github.com/voc/voctomix/tree/master/example-scripts
Run
Recent Version of GStreamer (1.6 and later)
Debian Stretch or Sid or Ubuntu Wily
or run a Docker
or build your own (uninstalled) GStreamer
https://packages.debian.org/stretch/voctomix
http://packages.ubuntu.com/yakkety/voctomix
https://hub.docker.com/r/c3voc/voctomix/
Talk
Questions?
froscon12-voctomix
By mazdermind
froscon12-voctomix
My talk about Voctomix for FrOSCon12
- 691