Writing an efficient
Logger in Go

Gopher: https://github.com/egonelbre/gophers

Who am I?

Damiano Petrungaro

Damiano Petrungaro

Italy

source: www.vidiani.com

source: www.greenparkmadama.it

source: www.milanotoday.it

Me everyday:

Learning

  • Learn to define idiomatic Go APIs
  • Learn to implement a safe and fast logger
  • Learn to benchmark and profile your code
  • Learn to write a faster JSON encoder

How to

  • Write a basic logger
  • Add benchmarks for the logger
  • Add APIs to add fields to logs
  • Write logs as JSON

  • Address JSON unmarshal failure

  • Add APIs to add trace IDs to logs

Write a basic logger

  1. Add logger interface
  2. Add tests
  3. Add logger implementation

Add benchmarks to the logger

  1. Add benchmarks for the logger
  2. Run and analyze memory profile
  3. Run and analyze CPU profile

Add APIs to add fields to log

  1. Add `With` method to interface
  2. Add tests
  3. Add benchmark: map vs slice
  4. Add slice implementation
  5. Add `With` logger benchmarks
  6. Check benchmark and profile delta

Write logs in JSON

  1. Change tests
  2. Change logger implementation
  3. Add other libraries to the benchmark
  4. Check benchmarks and profile data
  5. Break the API adding an invalid `any`

Address JSON unmarshal failure

  1. Add bug as test case
  2. Add APIs for a safer logger
  3. Check benchmarks and profile data
  4. Improve implementation if needed

Add APIs to add trace IDs to logs

  1. Add implementation as a decorator
  2. Change tests
  3. Change logger implementation

Recap

  1. We wrote an idiomatic logger
  2. We based our decisions on data
  3. We can add more decorators
  1. We could try another field implementation
  2. We should pass a real-life writer (os.Stdout)
  3. Visit the library damianopetrungaro/golog

Next steps

Useful links

  1. Github source code
  2. Profiling data visualizer (pprof)

Twitter: @damiano_dev
LinkedIn: /in/damianpetrungaro
Email: damianopetrungaro@gmail.com

Writing an efficient Go logger

By Damiano Petrungaro

Writing an efficient Go logger

Writing a logger can be tricky, and performance issues are hidden and tricky to find. Since I wrote a logger in Go, I want to share the knowledge I gained out of that by coding a session (inspired by the open-source one I wrote) to show bottlenecks from a performance and design perspective. The deep dive will involve writing benchmarks for encoding the log entry and profiling the whole logger design to determine where it could be improved.

  • 708

More from Damiano Petrungaro