"Domains"

The players

  • Console
  • Orchestration/Workflow
  • Restify
  • Config repo
  • Auth
    • Tier 1
  • Licensing
  • (Logs)

Tier 1

Login screen

Reverse proxy

Tokens

"Domains"

Tier 1

  • Enforce authentication before proceeding​
    • i.e. Request must have authenticated token before proceeding
    • Provides login screen
    • Does not implement authentication logic, forwards login request info
  • Enforce licence before proceeding
    • Could implement or not (i.e. could be another service)
  • Reverse proxy with rules
  • Implements domain switching
  • nab.mintpress.io -> Tier 1

Tier 1

Existing auth infra

Bundled auth provider

Auth admin interface

Auth

Auth

  • Authentication and Authorisation
    • Decoupled
  • The Licence impl (or separate service hooked in)
  • Must: pluggable providers
    • Ldap: Authentication and groups (for authorisation)
    • Internal datastore
      • Mgmt interface (gui or cli?)
      • Authentication and authorisation
        • Add authorisation info to externally authenticated requests (eg, ldap+internal)
    • Scope for future providers
    • Tokens
      • ↓

Tokens

  • Light vs heavy
    • Unnecessary complexity?
      • Facilitates user-facing (FE) token vs internal-facing (BE) token
  • User info
    • Profile
    • Authorisation
    • Sec groups/teams
  • "Domain" specific
    • i.e. Accessing a new domain is a new token
      • Only a new BE token
  • Licence information

Tier 1 and Auth

  • Tier 1 presents login screen if unauthenticated
    • Submits login form to auth for confirmation
    • Receives token in response
  • Subsequent requests from user to Tier 1 will provide the token
    • HTTP2 mitigates size; or use FE vs BE tokens
  • With a cache, Tier 1 confirms token is still valid (not blacklisted) with Auth
  • Tier 1 forwards Token to any backend service
  • Phase 1: Auth implements Licence logic

UI

  • User MGMT
  • Authorisation MGMT
  • Future:
    • Session awareness and token/session blacklisting
      • Blacklist all tokens for user X, etc

Tier 1

Console

Auth

Traffic flow

Reiterating: Token now available in Console:

  • User info
  • Licence info
  • etc

All traffic literally passes through the Tier 1.

Some requests reconfirmed with Auth (cached).

Console, add:

  • Needed UI functionality from RT:
    • Logs
      • Leverage a log server
    • Job control
      • Cancel underway, restart, etc
    • More job config?
    • What else?
  • Secret mgmt (access etc)
    • Phase 2, existing JSON for phase 1
  • Property config interface for Config Repo

Console, remove:

  • Explicit Auth:
    • Spring Security Compatible plugin which reads the token
    • Facilitate forcing a logout via response headers
  • Licence verification logic
    • Licence info available in token
  • GenericProperty et al
  • Console storage requirements are now UI related

Tier 1

Console

Auth

Config repo

Config Repo and Console

  • Using Config Repo for storing real config data
  • Move Service Catalogue Instance Definitions (SCIDs) to Config Repo
    • Facilitate export and import from Config Repo for multi-domain
      • Make it easy to automate
  • Config taxonomy:
    • Env
    • Provider
    • Instance
    • "Domain" is above all of this, not within it: i.e. run a new Config Repo for a new "domain"
  • Probably a REST-like interface
  • Need a facility for "migrations" for SCID metadata

Config Repo API

  • "Top level" REST-like API
  • Request must contain token
  • Used by Console
  • Available to users via Tier 1
    • Direct access available for internal needs

Things we should do better

  • Migrations
  • Import and export
  • (Marketplace)

Tier 1

Console

Auth

Config repo

Orch

Orchestration/Workflow

  • Config data stored in Config Repo
  • Phase 1: "text only" interface
  • Phase 2: GUI-ish interface, actually available in Console. Possibly implemented in Console and only uses APIs
  • REST-like APIs
    • Token must be provided
    • Publicly available
  • Logs
    • Forwards logs to log storage service
    • Could store internally
  • Nesting of units of work

Workflow language

osb -> security: "depends"
security -> oid: "depends
corebanking -> security: "depends"
corebanking -> osb: "depends"

↓

Workflow language

corebanking -> osb
osb -> security
corebanking -> security
security -> oid
security -> __emit__: "security done"

(`__emit__` is an idea for multi-orch interdependence, for perf...)

(Browser using graphviz via emscripten)

(We can enhance this with "depends" and other language)

Tier 1

Console

Auth

Config repo

Orch

Restify

HTTP2

Restify

HTTP(2)

SSH

Orch and Restify

  • Token provided
  • Orch talks to one or many Restifies.
  • Orch and Restify talk HTTP2
  • Restify can talk HTTP2 or SSH
    • Facilitate pluggable protocols
  • Restify can act as a proxy
    • Collocate a Restify with Orch and forward everything to a remote Restify (who is the actual master) via SSH if there are firewall issues
      • TCP PROXY protocol
  • Use service discovery to facilitate Restify and Orch finding each other

Restify

  • Invoke commands locally
  • Invoke commands via SSH
  • Cache logs until successfully sent to Orch
    • Log location can be configured:
      • STDOUT
      • /var/log/xyz.log
  • (Phase 2): use a service discovery framework  and create a Restify client (could be the same as the server?) to facilitate "bring up a new unmanaged X that self-registers"
    • Consul

Tier 1

Console

Auth

Config repo

Orch

Restify

Logs

Orch proxies the Logs UI (eg Kibana) to make it available to console.

API availability as well for per-job interface.

Logs

  • Use fluentd or Logstash or some such
  • Use Kibana or some such

Tier 1

Console

Auth

Config repo

Orch

Restify

Logs

"Domain"

"Domains"

  • Licensed feature?
  • Tier 1 and/or Auth need to be configured to know where the domain is
  • Where collocated, use Unix Domain Sockets for comms
    • /environmint/mintpress/domainX/ sockets:
      • console
      • config-repo
      • orch
      • restify
    • Hence, point Tier 1 and Auth to /environmint/mintpress/domainX/, and it knows where everything is​

Non-collocated

  • Facilitate configuring each service end point manually, default to consistency (as described above)
  • Tier 1 -> "Domain" proxy
    • https://host1/mintpress -> Tier 1 -> Tier 1 proxy on host2 -> unix:/environmint/mintpress/domainX/*
    • Make the proxy something simple like TCP PROXY or socat and allow piping it over sshuttle/ssh or similar

UI

  • When only one domain, domains are invisible
  • When disabled, domains are invisible
  • Domain switcher in top corner near logout
  • When within a domain, other than the switcher, there is nothing that explicitly makes you aware because "you're inside this world"
  • User may have different roles and security groups for each domain
  • Allow specifying on login
    • I actually kinda don't like the "look" of this?

Tier 1

Console

Auth

Config repo

Orch

Restify

Logs

"DomainX"

Tier 1

proxy

hostX

Individual pieces can be separated, but config is more complex

Tier 1 proxy might run on both sides

Tier 1

Console

Auth

Config repo

Orch

Restify

Logs

"DomainX"

Tier 1

proxy

hostX

Chef Provisioning

Mintspec

Licence

Appdeploy

Tier 1 has proxy rules for a top level public /api endpoint.

e.g.

/api/spec

/api/logs

/api/orch

Drift server

Managing the monster

  • S6 for process management
    • Tooling on top to make the process simpler
    • (Basically create a symlink to start the service, remove it to stop it)
    • Start/stop the monolith as a whole
  • Unix Sockets to avoid port requirements
  • Config Repo for all live config storage
  • Sane defaults

Other thoughts

  • Data store for console for live data
    • Move live data into it's own service in the future
    • Use data store that facilitates:
      • `"snapshot fs" == "backup"`
      • `"archive fs" == "backup"`
  • Mintspec service
    • Behind Console
    • UI in Console
  • AppDeploy service
    • Phase 1 UI: Deploy status read only
    • _Could_ use a similar autographing UI to Orch
  • Unified UI for Config Drift in Console
  • ↓

Other thoughts

  • Use Smartstack for service discovery within a domain
    • Cross host auto discovery
      • Removes complexity of config
      • Adds complexity of running more stuff
    • Could use for multi-domain discovery too
    • https://github.com/coreos/zetcd
    • Can always do a simple version with Consul, or any such
  • osquery for live status stuff
  • Live data for SC states, not just hardcoded

Phase 1: Config Repo

  • 6/7 envs
    • Different patterns
    • Maintaining templates
  • Goal: 1 pattern
  • Step 1:
    • POC/Prototype using Config Repo with Console and RT

Notes

  • Migrations
    • Forward in config repo
      • Always allow users to move forward in versions of config repo
      • Schema versions in config
  • Saveable-ish config
  • Take current model to new Config Repo seamlessly
Made with Slides.com