are underrated and underused.
They are a gold mine of precious data from both an operational and business perspectives..
(haven't we heard that 12 trillion times before?)

logs are usually

or even worse,

when we actually want them to be

wait, but.. we're already doing

so.. what's wrong?

logs come in different formats

parsing them is sometimes tedious, and dangerous

you should be doing:



is not a complete solution.
It is a part of it.
logs change!

take care of the data. Let the Ops handle the infrastructure.
(In a very broad sense of course)

start the structuring process

on the application side.
don't let logstash do the hard work, unless absolutely necessary.


should be an important part of the logging pipeline.
A lot of thought and work should be invested in it.

the application should send

  • meaningful (to help you understand what you're reading),
  • necessary (to save on resources and debugging time),
  • relevant (to target potential problems),
  • structured (to reduce parsing stress)



not to write to files to reduce disk IOPS stress.

send over the network if you can.

usually you see...'trying...')

what could you possibly make of this?


AZ = get_az()
REGION = get_region()
event_uuid = create_event_uuid()

# assume timestamp and host are provided by logstash{ 
  "message": "Currently registering user Michael Sverdlik.",
  "version": __version__,
  "module": __module__,
  "file": __file__
  "context": {
    "what": "registration",
    "sub_what": "phone",
    "user_plan": "premium",
    "user_id": id,
    "time_to_register": "16",
    "country": "NL",
    "attempts": "912",
    "region": REGION,
    "availability_zone": AZ,
    "event_uuid": event_uuid,

write log Messages

so that they're human readable.
{"level": "INFO", "message": "hacking user #USER#'s bank account #NUMBER#...", ,...,}
{"level": "DEBUG", "message": "retrieving the user's credit card and marital status.", ,...,}
{"level": "DEBUG", "message": "sending card number #NUMBER# to hacker #WHOIS#.", ,...,}
{"level": "DEBUG", "message": "waiting for hacker #WHOIS# to answer...", ,...,}
{"level": "DEBUG", "message": "hacker confirmation #NUMBER# received.", ,...,}
{"level": "DEBUG", "message": "sending the confirmation code to the DB.", ,...,}
{"level": "DEBUG", "message": "asking the DB for theft confirmation", ,...,}
{"level": "INFO", "message": "I managed to steal the user's money. Now go away!", ,...,}


that even if now, you're just viewing your log files, or even creating

you might end up analyzing your logs automatically.

also remember,

that this isn't about the tools themselves.
It's about the design of the pipeline -
and devs should be a part of that!

log aware development

By Nir Cohen

log aware development

  • 1,275