Loading

Symfony2 with heka

Abdoul N'Diaye

This is a live streamed presentation. You will automatically follow the presenter and see the slide they're currently on.

Abdoul N'Diaye

@AbdoulNDiaye

http://abdoul-ndiaye.info

Symfony Application's Monitoring with Heka

About Me

  • Symfony Lead Developer
  • Mostly PHP
  • Started SF since 1.4

Let's Get Started

Why collecting data is important?

Because :

  • Keep a visibility in complex applications
  • Detect potential errors
  • Improve your reactivity
  • tail is not a solution for everything :)

Logs are full of information BUT they are also full of useless

things when you have a problem to solve !

And now, what do you use ?

http://slides.seld.be/?file=2013-12-13+Application+monitoring+with+Heka+and+statsd.html#5

Nothing ? :(

System monitoring ?

collectd, cacti, nagios, ganglia, munin, ...

Performance / Metrics ?

 

Statsd, sentry, librato, ...

Error reporting ?

New relic, ...

Log aggregation and routing?

graylog, loggly, splunk, logstash, syslog, ..

+ Kibana, graphite, graphana, etc ...

Consequences....

To many tools

Not enough coffee

All tools have one pattern in common 

  • Get data
  • Transform data
  • Deliver data

One Solution

Heka

Made by Mozilla

Written in Go and Lua

Inspired by Logstash

What is already integrated in Heka?

System monitoring !

Performance / Metrics !

Log aggregation and routing !

Usefull dashboard

Basic Pattern

Inputs

Decoders

Router

Outputs

In details...

  • Simple to use/debug
  • Very fast
  • Heka can be extended by plugins in Lua

Heka is also :

Dashboard

Simple Aggregator

Getting started with Heka

v0.8.0

An example

sf2 app

heka

elastic search

Kibana

Main configuration

  • Choose your number of CPU’s Core available for Heka 
  • Activate the dashboard

Our goal

Centralise Symfony's logs

2 differents ways

First method

Keep the original file format of our logs

Advantages

  • Keep your original file format in the server
  • So, You have several ways to see your logs

Inconvenients

  • You have to configure each log file in heka
  • Files are not recommended with high traffic websites

Files

 

Log file configuration example

Configuration in Symfony

Second method

Send our logs via UDP ^^

Advantages

  • Performance
  • Less configuration in heka
  • Better management of logger channels

Inconvenients

  • No log if heka daemon is down

Explication Here

UDP

 

UDP file configuration example

Monolog.lua

Udp configuration

UDP : Symfony configuration

  • Less configurations in symfony
  • One input in heka
  • Less file opens by the system
  • File aggregation can make cry the server

And finally, add an ElasticSearch output

Conclusion

  • The most important is the strategy that you will choose after the installation of a log interface
  • Heka is not adapted for all case (little website)
  • You have to choose your best tools (they are all different)

Download heka in github : Here

Sample of init.d file : Here

Ansible role  : Here

Usefull Links

TomL language : Here 

Thank you

Questions ?

@AbdoulNdiaye

https://github.com/AbdoulNdiaye

abdoul.nd@gmail.com

Made with Slides.com