Observability?
Being able to ask questions about your environment without having to know ahead of time what you wanted to ask
-
What the heck was going on at 10:32 today?
-
What was the average response time for an API route last Tuesday?
-
Why did request 388abe6a spend so long querying the DB?
-
How many people are currently logged in?
-
What's the most popular page?
-
Where does all this traffic come from? Is it legit?
Why Observability?
Delayed::Job
https://w3.nue.suse.com/~lrupp/Presentations/OBS-Network/OBS-Network.svg
Application
- Modularity
- Coupling
- # of Codepaths
- Cyclomatic Complexity
- Algorithmic Complexity
- Architectural Complexity
- # of Libraries
Other
- Size (4.241 files, 722.517 LOC)
- Age (4.797 days a.k.a. 13 years)
- # of contributors (100)
- rate of change (26.156 commits, ~6/day)
It's the complexity, stupid!
How Observability?
A system is instrumented well enough if you can ask any question about it, required to support it at the level of quality you want to deliver to your users
Not having to add new instrumentation to get your new question answered
What to instrument?
High Cardinality
Events
- Who’s talking to our app?
- What are they asking of our app?
- How did our app deal with the request?
- Business-relevant data
- Additional context about the environment
How to instrument?
- As fast as possible
- Breadth-first and depth-first
- Easily Query-able
What can we do?
- Learn about observability, teach each other
- Treat owning your code and instrumenting it as a first class responsibility, and as a requirement for shipping it to production
- Find tools that let you do that easily
Learn More
- https://www.youtube.com/watch?v=1wjovFSCGhE
- https://charity.wtf/tag/observability/
- https://www.honeycomb.io/blog
- https://martinfowler.com/articles/domain-oriented-observability.html