Logging with Docker

faisal@druva.com

What's Docker?

  • Containers for app packaging

  • Like “chroot” on steroids

  • Image shipped as FS layers + metadata

  • Container runs with COW FS layer on top of image

  • FS layers repository for distribution

  • One whole copy of OS runs inside each VM.
  • Contains just App and dependencies
  • Runs in userspace in host OS

Virtual Machine

Docker

Docker Internals

Why Docker?

  • Change in paradigm
  • Transition from monolithic design to micro-services
  • Designed for scale
  • Easy to add new micro-services
  • Dev-ops model - predictable/repeatable services as containers

The downside

  • Application is collection of micro-services
  • Potentially different logging mechanism used by each service
  • Application logs all over the place
  • Log files need special handling to get out of the container
  • Monitoring applications/containers is challenging

Log/Monitor Strategy

  • Application spread across machines
  • Different logging mechanisms in use
  • Strive to consolidate all logs and metrics centrally
  • Analyse logs to make decisions about application health and scale
  • At scale, powerful log analysis absolutely necessary 

Log Targets

  • Log to file(s)
  • Log to stdout
  • Log to syslog
  • Log to custom logger
  • etc.

Log to file(s) on Docker

  • Map common volume(s) for logs into each container
  • Volume(s) can be docker-only or host-mapped
  • Run a collector on the docker host or in container to send collected logs to central logging

Log to stdout

  • Recommended by Docker
  • Easy to develop for
  • May not work for existing apps
  • Offload to file/data store
  • logspout
  • Use log driver

Log to stdout

Log to syslog

  • Good support under Linux
  • Multiple ways of running under docker

Syslog on host

  • Set up logger daemon on host as usual
  • Map /dev/log on host to /dev/log on all containers when launching
  • Log post-processing/analysis on host machine

Syslog on host

Syslog in container

  • Logging container started first
  • /some/host/path mapped to /dev for this container
  • /some/host/log/output mapped to /var/log
  • Each app container launched with /some/host/path/log mapped to /dev/log

Syslog in container

Docker logging driver

  • Application logs to stdout
  • Log backend can be changed without changing app
  • Supports:
  1. json-file: default
  2. syslog
  3. journald (systemd)
  4. gelf (Graylog Extended Log Format)
  5. fluentd
  6. awslogs (AWS cloudwatch logs)

Docker logging driver

Example: syslog driver

  • Great feature for non-syslog capable apps
  • Leverage existing logger
  • Usable with logger running on host or in a single container:
$docker run --log-opt syslog-address==unix://path ...
  • Or on a remote host:
$docker run --log-opt syslog-address=[tcp|udp]://host:port  ...
Made with Slides.com