Distributed Computing in Media Arts
Andrés Cabrera
CREATE Research Director
AlloSphere Software Infrastructure Engineer
Dec 4 2017
Distributed computing
- The "application" consists of more than one process
- Often these processes run on independent hardware
e.g.
- Web Apps/Mobile Apps
- Super computer simulations
- Including 3D render farms
- Anything with a client/server or P2P arch
- In media arts:
- Supercollider/PD
Network performance
- Audio, video and control over the net
Distributed interfaces
- Distributed interfaces for control
Collaborative live coding
- Gibber
Most efforts have centered around distributed interaction
Distributed computing is easier than ever
Cuttlefish
Easier
- Lower price of hardware
- Field of distributed computing well studied
- Many tools are available
But it's still hard
Why is it hard to make?
- Multiple applications/multiple devices
- Data exchange
- Synchronization
- Tedious prototyping workflow
- Must build/deploy to multiple devices
Why is it hard to run?
- Deployment
- Bottlenecks
- Network bound
- Processing bound
- Data I/O bound
Why do it?
- Super powers
- Many users/controllers
- Distant collaboration
- Big data
- Large/RT machine learning
- Higher resolution
- Lower latency/lag
Practical Models
Architectures
- Client-Server
- Many identical clients connect to a server
- Typical scenario for multi-user interaction
- "Master" application
- One application is the master
- The rest listen to instructions by the master
- (but the master need not be aware of them)
- Peer-to-peer
Interapplication communication
- Synchronous state
- Asynchronous event propagation
State Broadcast
- Known fixed size
- Static
- States are independent to each other
- And provide all the information needed by the receivers
Event distribution
- Discrete asynchronous information from a master controller
- Master may or may not know information about the listeners
Master-Nodes
- Nodes register with a master
- Master sends discrete information to the involved nodes
- Useful for dynamic systems.
- The nodes are notified by the master when they need to create "stuff"
Peer-to-peer
- Each node connects to another freely
Speeding up computation
Parallel vs. Serial
Parallel
- Concurrent
- Models:
- SPMD/SIMD (Data parallelism)
- MISD (Task parallelism)
- multi-core/multi-threaded/cluster
- Requires thought
Parallel (Macro to Micro)
Macro
Parallel (Macro to Micro)
Micro
GPU shaders
GPU Core
GPU Core
GPU Core
...
Shader program runs on all cores
Pixels, vertices, etc.
Data
Parallel (Macro to Micro)
Micro
Thread pool
Thread
Thread
Thread
...
All threads execute the same function
Data
Parallel (Macro to Micro)
MISD (Multiple Instruction Single Data)
Data
Process 1
Process 2
Process N
...
All outputs are different
Parallel examples
- High audio channel count
- Wavefield
- Ambisonics with large number of sources
- Multiple perspectives of a scene
- Complex post-processing
- Shaders are parallel
- Independent rendering and compositing
- Equalizer
Serial
- Processes depend on previous results
- Only really useful in real-time
- Trading latency for processing power
- A lot of audio DSP is serial
My example (Nyia)
My example (Nyia)
- 5 different applications:
- Renderer
- Simulator
- Audio (x2)
- Control
- Over two network layers
- 10G and 1G
Parts
Simulator
Renderers
Control
Audio (x2)
1G events
10G State bcast
localhost - events
Ideas
- Tactile control pumps "chaos" into the system
- Finger position is sent to simulator
- Simulator determines current value of chaos and distributes it
- Each node determines a lot of actions from this value
- The simulator determines when a hex pair is generated and sends information to its dynamic nodes (control and renderers)
AlloSystem
- Served as the framework for all applications
- Cuttlebone for efficient state broadcast on the 10G network
- Developed distributed dynamic master-node classes for graphics
- Listener model
- Python scripts for concurrent building and deployment
What is distributed processing anyway?
- What's the difference between moving data and processing between two threads, a CPU and GPU, and moving it between a computer and a mobile phone?
Another take (Stride)
- A language for sound synthesis, processing, and interaction design
- Declarative and reactive
- Very minimal syntax
- Abstracts hardware
- All hardware is a signal/event generator
Hardware Abstraction in Stride
- Hardware I/O is abstracted at a high level
- Flow of data across different computation domains is abstracted
- Whole systems and movement of data across them can be abstracted in the same way
Stride Systems
- Systems in Stride are a list of:
- Hardware + Platform
- Connection protocols between them
use ArduinoAndPC version 1.0
signal Frequency { domain: ArduinoDomain }
AnalogIn[1] * 440 / 1024 >> Frequency; # On Arduino
Oscillator(frequency: Frequency) >> AudioOut[1]; # On PC
Stride Systems
use AlloSphere version 1.0
AudioIn[1] >> SpeechRecognition() >> UserName;
[UserName, "Andres"] >> Equal() >> EverythingOn();
reaction EverythingOn {
streams: [
on >> ResetInterface();
0 >> CurrentPreset;
0.0 >> CurrentTime;
-20.0 >> LevelDb;
]
}
Much to be done
- Infrastructure
- We need experts in audio, graphics and interaction to drive the development of large scale libraries
- But a lot is possible now.
Thanks!
Questions?
Distributed Computing in Media Arts
By mantaraya36
Distributed Computing in Media Arts
- 966