Architecture Overview

1.  Agent 'Seascape Wave'.
2.  Ingest listening/computing nodes.
3.  Analytics-focused distributed database (ElasticSearch).
4.  Web user-interaction nodes.

Four key components:

Architecture Diagram

Contents

1. Architectural Overview
2. The Seascape Wave Agent
3. The central elixir application
4. Conclusion

Agent: 'Seascape Wave'

Agent: 'Seascape Wave'

• Runs on a centralized machine where metrics can be aggregated.
• We want to construct the system to easily allow adding other agents in the future (might run inside environments without access to the host machine).
  • Start with saltstack
  • Allow for further expansion

Agent: 'Seascape Wave'

• Built using Python/SaltStack
  • Built-in cryptography
    • Verified identities
    • Encrypted communication
  • Built-in message queue
• Runs on the 'Salt master' which manages the container hosts (minions):
  • Minions can be configured by the agent to send metrics to the master. The agent can then format and send them to the ingest nodes.
  • Agent must have access to the machine hosting the containers.

Contents

1. Architectural Overview
2. The Seascape Wave Agent
3. The central elixir application
4. Conclusion

Ingest nodes

• Built using Elixir.
• Part of the BEAM cluster (see later slide).
• Horizontally scalable by running many behind a load-balancer (distributing sessions in a round-robin fashion).
• Listens for a data from an Agent.

Analytics-focused distributed database

• For this task we've chosen to use Elasticsearch.
  Reasons:
  • Built to work with metric-based data like what we have in this project.
  • Built to scale to large mountains amounts of data.
  • Built to refine the interpretation of data (e.g what kind of queries you might want to do) later.

Web nodes: construction

• Built using Elixir/Phoenix LiveView.
  • A user connects to it with their browser.
  • LiveView serves a static HTML page containing the SPA on first visit which then starts up a WebSocket connection through which all later interaction takes place:
    • DOM-diffing happens server-side, simplifying implementation.
  • Does not interfere with SEO, for example
• Part of the BEAM cluster (see later slide).

Web nodes: scaling

• Part of the BEAM cluster (see later slide).
• Horizontally scalable by running many behind a load-balancer (distributing sessions in a round-robin fashion).
• Requests data from the database whenever required, caches data locally.

The BEAM cluster: actors

• Elixir works using the Actor model, allowing for a scalable and fault-tolerant implementation of concurrent services that is comprehensible and therefore maintainable.

The BEAM cluster: actors

• Ingest and Web nodes are connected to the same Erlang/BEAM cluster, enabling:
  • Transparent message-passing, in particular:
  • Transparent pub-sub for real-time data updates which can be sent without going through the database.
  • CRDT-based real-time information about what agents (e.g. the user's clusters and containers within them) are currently 'up' and their health.

The BEAM cluster

• On top of this, we run the distributed database Mnesia on all nodes in the cluster, to enable:
  • Persistent user sessions (without complicating the load balancers).
  • Storage of ephemeral (e.g. current 'real-time' data).

Contents

1. Architectural Overview
2. The Seascape Wave Agent
3. The central elixir application
4. Conclusion

General Remarks

• For our particular project, there is no functionality we can provide to the user when they are offline. (Except telling them that they are disconnected and that the current metrics/graphs thus are stale.
  • This in part influenced our design decisions.
• We are not adding extra 'message queues' (like e.g. RabbitMQ) to our stack, besides the message queue that is part of the Agent (internally in Saltstack in our case).
  • Of course, all Elixir processes (actors) have their own internal message queue ('mailbox').
    Fun fact: RabbitMQ is itself written in Erlang and thus uses these techniques under the hood
• We are not adding external caches/message brokers (like e.g. Redis) to our application. Instead, we use direct message passing instead of an external message broker and Mnesia for persistent user sessions and ephemeral caching.