Real-time KPI metrics with Graphite, StatsD and Django
Israel Saeta Pérez (@dukebody)
measure anything, measure everything
SQL is not the way to go
StatsD
Application
Graphite
Dashboard
Alerting system
architecture
Graphite
All-in-one solution for storing and visualizing time-series data in an efficient manner
Whisper
Fixed-size time-series numeric database
carbon
Metric receiver, aggregation and disk flushing
Graphite web
Django app for rendering graphs and pulling arbitrary data
basic config
[min] pattern = \.lower$ aggregationMethod = min
storage-schemas.conf
storage-aggregation.conf
[my_schema] pattern = .* retentions = 10s:6h,1min:6d,10min:1800d
basic usage - easy!
http://graphite.host/render?target=metric.name
Send metric
<any.metric.name> <number> <timestamp>
stats.coffee.left.cups 5 1434305622
get graph
Dozens of yummy transformation functions
- Averages, max/min, percentiles...
- Aggregate by time window (ex. hits per minute)
- Derivative, integral, power, scaling, differences
- Time warping
- Check the docs!
Also JSON and CSV
Feed your own app!
statsd
Network daemon that listens for statistics and send aggregates to pluggable backend services
Uses UDP: fast, fire-and-forget
Client libraries in all popular languages
Main metric types: counter, timer, gauge
<metricname>:<value>|<type>
echo "foo:1|c" | nc -u -w0 127.0.0.1 8125
pystatsd
Python client for StatsD
>>> import statsd
>>> c = statsd.StatsClient('localhost', 8125)
>>> c.incr('foo') # Increment the 'foo' counter.
>>> c.timing('stats.timed', 320) # Record a 320ms 'stats.timed'.
pip install statsd
django-statsd
Convenience wrapper to use pystatsd in Django
>>> from django_statsd.clients import statsd
>>> statsd.incr('python.meetup')
pip install django-statsd-mozilla
awesome goodies!
- StatsD config in settings.py
- Requests, database, cache and frontend timings
- Models create, update, delete signals count
- Celery signals
final notes
sudo docker run -d --name graphite \
-p 80:80 -p 2003:2003 -p 8125:8125/udp \
hopsoft/graphite-statsd
how do i install all this???
graphite graphics are fugly!
Use Grafana
graphite components are too slow!
ALTERNATIVES
Riemann.io + InfluxDB + Grafana
Prometheus.io
StatsD
Riak
the end
Real-time KPI metrics with Graphite, StatsD and Django
By Israel Saeta Pérez
Real-time KPI metrics with Graphite, StatsD and Django
How to use StatsD and Graphite to do application and KPI monitoring.
- 4,222