Log Aggregation with Graylog

Why Log-Aggregation?

  • Accessibility
  • Usability
  • Quantitative analysis
  • Alerting

Graylog Dashboard

Graylog Features

  • Open Source (Open Core)
    • Optional Enterprise Features
  • Collection & Extraction
  • Interactive analysis and searches
  • Dashboards & Visualization
  • Alerts & Triggers

Architecture

Architecture (Cluster)

Architecture (Kafka + Fluentd)

billions of logs per day, > 100k messages per second 

Transport

  • Beats
  • GELF
  • Logstash
  • Kafka
  • Fluentd
  • All kinds of crazy combos! 

Application Logging

  • Logback driver
    • STDOUT
    • special driver, i.e. GELF
  • Docker
    • default (JSON file)
    • GELF
    • journal
  • Sidecar containers possible

Beware Multiline Java-Stacktraces!

Use Logstash or GELF appender
¯\_(ツ)_/¯ 

multiline {
  #type => "all" # no type means for all inputs
  pattern => "(^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)"
  what => "previous"
}

Exercise

  • Checkout the workshop and get Graylog up and running
    • https://github.com/kiview/graylog-workshop
  • Configure Docker GELF driver in order to let containers send logs to Graylog
  • Configure Graylog extractors
  • Setup some useful streams and visualizations
Made with Slides.com