LogRhythm Platform

Services For Business Domains

• Oriented around specific logical domains
• Data is accessed and mutated only through services
• Deal in data, not interface

Deployment Of Services

• Deployed across hardware & the network in any way
• APIs should be treated as public facing at all times
• APIs should give a consistent interface and support multiple transport layers

The Business Case

Granularity

• Scale at the bottleneck
• Sizing of deployments for sales
• Services as sellable features

Development

• Better use of dev teams and their expertise
• Protects legacy code from risky change
• Technical decisions matter to investors

MicroServices

The Pros and Cons

Pros

• Segregate domains and enforce single responsibility
• Network as the seam
• Choose the right tool for the job

Risks

• Micro Silos
• More smaller processes
• Cross service joins of data

Guidelines

Some lessons learned!

Team Process

• One service = one runtime process, one repo
• No flying solo
• No cute names!

Deliverables

• Docker, vagrant, chef scripts, oh my!
• Nevermind. Just make an installer.
• Create self-contained binary packages.

Categorizing Services

• Stateless
  • API Services
  • Processing Services
• Stateful
  • Persistence layers
  • Ephemeral state tracking

The Runtime

• Use a proc manager
• Disposable processes
• Write logs to stdout
• Take config from env variables

Fault Tolerance

• Perform graceful shutdown on SIGTERM
• Do not break on non-graceful shutdown
• Fault tolerant of upstream dependencies

I swear it's real!

Consul!

• One local agent on each machine
• Agents are configured with other locations at install
• Services register, deregister, and heartbeat
• Services request dependencies through consul

What emerges?

• No more location configurations
• Shared almost nothing - localhost:8500
• Visibility into the state of the system
• NRT reaction to changes in system state