Monitoring microservices
What is this for?
To know what is going on with our running services.
How many users
% errors per requests
Is it running
Average time of response
# containers registered
# messages processed
error rate on B2B
General metrics:
Custom metrics:
Why is important?
It will give us information on how good or bad is our system performing.
We want to know before our customers know about an issue.
It gives us visibility on where to focus and alert us.
How does it work?
Micrometer
Actuator endpoints output metrics in a specific format that is readable for prometheus.
Micrometer exposes this data from our JVM and some automatic gather information. eg http calls
Prometheus
Polls every x seconds to each service to get this data in a specific format.
(like a file)
Prometheus can discover services from services like (consul, kubernetes, etc) so no hardcoded targets are needed.
AlertManager
Listen to alarms triggered by Prometheus configured rules and uses registered channels to notify (slack, pager, email, etc)
It has features like silencing an alert, promoting, non-repeating, etc.
Grafana
Grafana can read from multiple sources and offers better graphs and user management.
It just reads from Prometheus and uses PromQL to set metrics.
DEMO!
Lets see it in action
How do use it?
- Clone from PoC repo your own and play with PromQL
- Create your own metrics and alerts.
- Load it with lots of requests and see how the graphs behave.
From next sprint:
- Use your credentials to see mawani dashboard and create your own to see data :)
Pending:
- Deploy to core-stg using ansible scripts.
- Deploy to kubernetes cluster using deployments.
- Define credentials for grafana.
- Make PRs for micrometer
- Make PRs for securing data points.
- Create TOKEN for data access only.
Links:
Learn PromQL:
https://medium.com/@valyala/promql-tutorial-for-beginners-9ab455142085
Create your own data points:
https://www.atlantbh.com/blog/custom_metrics_micrometer_prometheus_spring_boot_actuator/
Questions?
Monitoring microservices
By Alejandro Vidal Rodriguez
Monitoring microservices
Micrometer + prometheus + grafana + alertmanager
