Messaging
The current state of component communication
- core/ml: python bindings
- core leader/follower: database with driver
- core/edge: nodes js plugin, rpc calls through the node js plugin
- core/argus: writing files on disk, sending metrics through rpc
- leader c++/python: named pipe
=> Custom solution for each, complex, not scalable, hard to deploy
A possible solution: messaging
- Decouples each components
- They just need to worry about ins, outs and transform
- Can all be written in different languages
- Easy to test, monitor, modify, scale
- Each component can run anywhere they want, just need a connection to the broker
- Performance of those more than enough
- One way to rule them (almost) all, simplifies maintenance
- Needs broker process/machine(s)
Example workflow
SCT (no more node JS plugin)
Poi
Edge
Standalone
Broker
Example workflow
MCT (no more JS plugin, no more two leader processes)
Poi
Edge
Standalone
Broker
+
data
Poi
Standalone
Poi
Poi
Standalone
Leader
Poi
Poi
Poi
Nats (https://nats.io/)
- Open source (licence Apache 2)
- Lightweight and performant
- Simple to use and setup (both for developers and ops)
- Drivers in the languages we need (Node Js, C, Python, and more)
- Comes out of the box with all the tools we need:
- Transmit bytes
- Any number of publishers and subscribers
- Secure connection to server
- And that we will need:
- Scale
- Persist data
- ...
Messaging
By technology
Messaging
- 44