Distributed Computing in Media Arts

Andrés Cabrera

CREATE Research Director

AlloSphere Software Infrastructure Engineer

Dec 4 2017

Distributed computing

  • The "application" consists of more than one process
  • Often these processes run on independent hardware

e.g.

  • Web Apps/Mobile Apps
  • Super computer simulations
    • Including 3D render farms
  • Anything with a client/server or P2P arch
  • In media arts:
    • Supercollider/PD

 

Network performance

  • Audio, video and control over the net

Distributed interfaces

  • Distributed interfaces for control

Collaborative live coding

  • Gibber

Most efforts have centered around distributed interaction

Distributed computing is easier than ever

Cuttlefish

Easier

  • Lower price of hardware
  • Field of distributed computing well studied
  • Many tools are available

But it's still hard

Why is it hard to make?

  • Multiple applications/multiple devices
  • Data exchange
  • Synchronization
  • Tedious prototyping workflow
    • Must build/deploy to multiple devices

Why is it hard to run?

  • Deployment
  • Bottlenecks
    • Network bound
    • Processing bound
    • Data I/O bound

Why do it?

  • Super powers
  • Many users/controllers
  • Distant collaboration
  • Big data
  • Large/RT machine learning
  • Higher resolution
  • Lower latency/lag

Practical Models

Architectures

  • Client-Server
    • Many identical clients connect to a server
    • Typical scenario for multi-user interaction
  • "Master" application
    • One application is the master
    • The rest listen to instructions by the master
    • (but the master need not be aware of them)
  • Peer-to-peer

Interapplication communication

  • Synchronous state
  • Asynchronous event propagation

State Broadcast

  • Known fixed size
    • Static
  • States are independent to each other
    • And provide all the information needed by the receivers

Event distribution

  • Discrete asynchronous information from a master controller
  • Master may or may not know information about the listeners

Master-Nodes

  • Nodes register with a master 
  • Master sends discrete information to the involved nodes
  • Useful for dynamic systems.
    • The nodes are notified by the master when they need to create "stuff"

Peer-to-peer

  • Each node connects to another freely

Speeding up computation

Parallel vs. Serial

Parallel

  • Concurrent
  • Models:
    • SPMD/SIMD (Data parallelism)
    • MISD (Task parallelism)
  • multi-core/multi-threaded/cluster
  • Requires thought

Parallel (Macro to Micro)

Macro

Parallel (Macro to Micro)

Micro

GPU shaders

GPU Core

GPU Core

GPU Core

...

Shader program runs on all cores

Pixels, vertices, etc.

Data

Parallel (Macro to Micro)

Micro

Thread pool

Thread

Thread

Thread

...

All threads execute the same function

Data

Parallel (Macro to Micro)

MISD (Multiple Instruction Single Data)

Data

Process 1

Process 2

Process N

...

All outputs are different

Parallel examples

  • High audio channel count
    • Wavefield
    • Ambisonics with large number of sources
  • Multiple perspectives of a scene
  • Complex post-processing
    • Shaders are parallel
  • Independent rendering and compositing
    • Equalizer

Serial

  • Processes depend on previous results
  • Only really useful in real-time
    • Trading latency for processing power
  • A lot of audio DSP is serial

My example (Nyia)

My example (Nyia)

  • 5 different applications:
    • Renderer
    • Simulator
    • Audio (x2)
    • Control
  • Over two network layers
    • 10G and 1G

Parts

Simulator

Renderers

Control

Audio (x2)

1G events

10G State bcast

localhost - events

Ideas

  • Tactile control pumps "chaos" into the system
  • Finger position is sent to simulator
  • Simulator determines current value of chaos and distributes it
    • Each node determines a lot of actions from this value
  • The simulator determines when a hex pair is generated and sends information to its dynamic nodes (control and renderers)

AlloSystem

  • Served as the framework for all applications
  • Cuttlebone for efficient state broadcast on the 10G network
  • Developed distributed dynamic master-node classes for graphics
    • Listener model
  • Python scripts for concurrent building and deployment

What is distributed processing anyway?

  • What's the difference between moving data and processing between two threads, a CPU and GPU, and moving it between a computer and a mobile phone?

Another take (Stride)

  • A language for sound synthesis, processing, and interaction design
  • Declarative and reactive
  • Very minimal syntax
  • Abstracts hardware
    • All hardware is a signal/event generator

Hardware Abstraction in Stride

  • Hardware I/O is abstracted at a high level
  • Flow of data across different computation domains is abstracted
  • Whole systems and movement of data across them can be abstracted in the same way

Stride Systems

  • Systems in Stride are a list of:
    • Hardware + Platform
    • Connection protocols between them
use ArduinoAndPC version 1.0

signal Frequency { domain: ArduinoDomain }

AnalogIn[1] * 440 / 1024 >> Frequency; # On Arduino

Oscillator(frequency: Frequency) >> AudioOut[1]; # On PC

Stride Systems

use AlloSphere version 1.0

AudioIn[1] >> SpeechRecognition() >> UserName;

[UserName, "Andres"] >> Equal() >> EverythingOn();

reaction EverythingOn {
    streams: [
        on >> ResetInterface();
        0 >> CurrentPreset;
        0.0 >> CurrentTime;
        -20.0 >> LevelDb;
    ]
}

Much to be done

  • Infrastructure
    • We need experts in audio, graphics and interaction to drive the development of large scale libraries
  • But a lot is possible now.

Thanks!

Questions?

Made with Slides.com