@lxsameer

  • More than 12 years experience
  • A free software guy
  • Contributions: Rails, Debian, Emacs, GNU, Pedestal, ReactNative, CLJSJS, etc

Work In Progress!!!

I just want to share my idea 

Topics

  • Brief history
  • Current implementation
  • Another framework ???
  • Goals

Story time 

Stock Exchange & Data processing

  • Collect data and metrics from devices and other markets
  • Analyze the collected data and provide information for customers
  • Serve real time data to the customers.

Our Requirements

(General goals)

Scalable

  • Scale up and down based on work load
  • Scale up based on latency 

Fault tolerance 

  • Against network failures
  • Against network partitions
  • Against human errors
  • Against hardware failures
  • etc

Consistancy

  • Reliable and correct information
  • Serve most recent data or nothing

CAP theorom and availability 

User's decision

Extendable 

  • Easy to understand
  • Easy to reason about
  • Maintainable 
  • As Simple as possible

Couple of other products with the same structure

That got me thinking...

Implementation

Current Implementation

HellHound = Clients + Web Cluster

Clients

  • Client as a component of system
  • It's different than other components
  • Re-frame application
  • Websocket connection 
  • Fallback to ajax ( removed )
  • A re-frame event which populates the app-db
  • Lots of subscribers

Why Websockets ?

  • Low latency
  • Small packet size
  • Browser support
  • Tooling and libraries

Why Not ?

Asset caching

Websocket Alternatives

  • QUIC
  • HTTP 2.0
  • SSE
  • SPDY
  • WebRTC data channels

Immutant Workers

  • Should be Stateless ( websocket )
  • Ring + Immutant
  •  Immutant websocket server
  • System 
  • Event Router

Kafka

  • Task Streams
  • Info Steams

Onyx Worker

  • Stateless
  • Lightweight

HellHound

Another web framework ????

Seriously ???

Why bother ? 

  • Extract a framework or library from the product
  • Improve the framework for future products

Ultimate goals

  • FullStack
  • Async
  • Distributed
  • Stream Processing
  • Collection of smaller subsystems
  • As simple as possible

Goals

Server Side

Problems & Improvements

  • Messaging Abstraction ( Manifold )
  • Workflow
  • Communication Abstraction
  • Server <-> Client events and event router
  • Back Pressure
  • Erlang Style Error handling per node
  • Environments

Changes

  • Aleph
  • Manifold

Other stuff which I tested

  • Pedestal
  • Yada

Client Side

Problems & Improvements

  • Data router  ( Ditch the reducers )
  • Query Language
  • App-db index 
  • Server <-> Client events
  • Service Workers
  • Web Worker support
  • Caching 
  • Preact and other minor tweaks 
  • Environments

So HellHoud would be

An opinionated fullstack framework to create async, distributed web applications based on data streams.

You should be able to focus on your business logic rather than the DETAILS of the distributed system, let other people worry about them.

Contribute

Stay in touch

  • hellhound.io
  • @lxsameer twitter and github
  • #hellhound on freenode and clojurians
  • lxsameer@gnu.org

Q&A

Thanks

Made with Slides.com