@lxsameer
- More than 12 years experience
- A free software guy
- Contributions: Rails, Debian, Emacs, GNU, Pedestal, ReactNative, CLJSJS, etc
Work In Progress!!!
I just want to share my idea
Topics
- Brief history
- Current implementation
- Another framework ???
- Goals
Story time
Stock Exchange & Data processing
- Collect data and metrics from devices and other markets
- Analyze the collected data and provide information for customers
- Serve real time data to the customers.
Our Requirements
(General goals)
Scalable
- Scale up and down based on work load
- Scale up based on latency
Fault tolerance
- Against network failures
- Against network partitions
- Against human errors
- Against hardware failures
- etc
Consistancy
- Reliable and correct information
- Serve most recent data or nothing
CAP theorom and availability
User's decision
Extendable
- Easy to understand
- Easy to reason about
- Maintainable
- As Simple as possible
Couple of other products with the same structure
That got me thinking...
Implementation
Current Implementation
HellHound = Clients + Web Cluster
Clients
- Client as a component of system
- It's different than other components
- Re-frame application
- Websocket connection
- Fallback to ajax ( removed )
- A re-frame event which populates the app-db
- Lots of subscribers
Why Websockets ?
- Low latency
- Small packet size
- Browser support
- Tooling and libraries
Why Not ?
Asset caching
Websocket Alternatives
- QUIC
- HTTP 2.0
- SSE
- SPDY
- WebRTC data channels
Immutant Workers
- Should be Stateless ( websocket )
- Ring + Immutant
- Immutant websocket server
- System
- Event Router
Kafka
- Task Streams
- Info Steams
Onyx Worker
- Stateless
- Lightweight
HellHound
Another web framework ????
Seriously ???
Why bother ?
- Extract a framework or library from the product
- Improve the framework for future products
Ultimate goals
- FullStack
- Async
- Distributed
- Stream Processing
- Collection of smaller subsystems
- As simple as possible
Goals
Server Side
Problems & Improvements
- Messaging Abstraction ( Manifold )
- Workflow
- Communication Abstraction
- Server <-> Client events and event router
- Back Pressure
- Erlang Style Error handling per node
- Environments
Changes
- Aleph
- Manifold
Other stuff which I tested
- Pedestal
- Yada
Client Side
Problems & Improvements
- Data router ( Ditch the reducers )
- Query Language
- App-db index
- Server <-> Client events
- Service Workers
- Web Worker support
- Caching
- Preact and other minor tweaks
- Environments
So HellHoud would be
An opinionated fullstack framework to create async, distributed web applications based on data streams.
You should be able to focus on your business logic rather than the DETAILS of the distributed system, let other people worry about them.
Contribute
- Free Software
- http://github.com/Codamic/hellhound
Stay in touch
- hellhound.io
- @lxsameer twitter and github
- #hellhound on freenode and clojurians
- lxsameer@gnu.org
Q&A
Thanks
HellHound
By Sameer Rahmani
