Anatomy of a Full Stack Project
Yuval Adam
- Full stack developer and consultant
- Open source, privacy & security
- Django Girls TLV co-organizer
- https://yuv.al
- @yuvadm
Project Overview
Stop Motion @ Jerusalem Film Festival
- Stop motion stations designed for children
- Hardware by Gil Adam and Jonathan Ward
Stop Motion
Stop Motion
From web app to real-life app
Block Diagrams
Stop Motion Stations
High Level Overview
Stop Motion Stations
Raspberry Pi Stations
System
- Arch Linux ARM
- Customizable
- Lightweight (raspbian is bloated)
- systemd units for running daemons
- systemd for network connections
- Auto-login && startx
- via ~/.bash_profile
- should have used systemd but was lazy
Image-based UI
API
- Movie package
- frames 0000.jpg, 0001.jpg, ....
- MANIFEST
- (NO)PUBLISH flag
- rsync all files to big screen unit for processing
- done
Scalability
- How do we preview at 10 frames per second?
- Low res thumbnails - in memory
- High res preview - overlay feh slideshow
- Long running UI process
- Check carefully for memory leaks!
- Worst case - process dies and restarts
Big Screen Processing
Processing Unit
- Receive movie packages
- Process *.JPG -> MP4
- Display on central screen
Processing Unit
- Odroid XU4
- 2 x quad core ARM
- 2GB RAM, Mali ARM GPU
- $80
Processing Flow
- Python-based daemon
- Watch for incoming movie packages
- ffmpeg
- compile 4 second title frame
- compile 10fps movie
- concat title + movie
- upload final assets to S3
- mpv
- control via UNIX socket commands
Scalability
- compile movies
- maintain a smooth display on TV
- ffmpeg is a monster, limit threads
Network
Router
- TP-Link WDR4300
- OpenWrt
- Open source Linux firmware for routers
- Uplink via 4G netstick
- WLAN coverage for all devices
Tunneling
- Remote access for maintenance and debugging
- Dynamic DNS good enough on landline
- No good on 4G
- OpenVPN tunnel
- Use local server for reduced latency
- Map chosen ports -> 22 on each device
- 2200X > 10.0.0.X:22
Security
- Enable SSH only from VPN/LAN, or:
- Disable password login (key-based ONLY)
- or you will get scanned and pwned
- Leave no port open #IoT
Website
Website
- Display all movies created each day
- http://stopmo.jff.org.il/
Website
- Simple Flask application
- Pull metadata from S3
- Stash into Redis
- All movies are served directly from S3
- in-browser or downloaded
- Deployed on Heroku
Poor Man's Background Task
@app.after_request
def update_data(response):
try:
if not redis.get('fresh'):
redis.setex('fresh', 'yes', 60)
run_expensive_update_operation()
except ConnectionError as e:
if app.debug:
run_expensive_update_operation()
return response
Full stack
- Prepare to take care of everything
- 'Secondary' tasks are now high-priority
- Logging
- Security
- Debugging
- Performance and scalability
- know your hardware
Questions?
Anatomy of a Full Stack Project
By Yuval Adam
Anatomy of a Full Stack Project
- 3,515