Why Logging is Important

...and how Logality can help you with that

By Thanos - @thanpolas

Node.js

Hi, I am Thanos

Nearly 50 published NPM packages.
Over 200 OSS repos.
Contributor in major and small OSS projects.
Retired Software Engineer turned Gardener 👨‍🌾.
Now developed a keen interest in DeFi.

twitter.com/thanpolas

Today's Menu

Why Logging Is Important
Logging Requirements
What and How to Log
Where and When to Log
How Logality Can Help
Other Logging Packages

Why Logging Is Important

Having a log at the right place, with the right kind of information can save your 🥓🥓🥓.

Logging Helps You With...

Why Logging is Important

Debugging.
Error Tracing.
Performance Troubleshooting.
Accounting.
Audit Trail.
Security.

Logging Requirements

Your Environment

Has multiple instances of your application.
Has multiple workers.
Probably has multiple stacks and runtimes.
Uses multiple services and databases.
Runs on multiple platforms.

Consuming Logs

Logging Requirements

Common Logging Aggregator.
Queryable Aggregator --> JSON Logs.
Common Logging Schema.

Consuming Logs

Logging Requirements

Common Logging Schema

Allow me to repeat this...

What and How to Log

Who am I?
Where am I?
What am I?
Metadata
Flags and Tagging

A Log Message Should Contain

Who Am I?

What and How to Log

What is my runtime? (Node.js?)
What hostname am I on? (abc.aws.com)
What is my process id? (10420)
What is my process name? (npm start)

Where and What Am I?

What and How to Log

What is my logging Level & Severity? (Info, Warn)
What Module is the log from? (app/service/db.js)
What function is this log from? (init())
What time is it? (2021-05-06T12:54:31.978Z)

Flags and Tagging

What and How to Log

In a JSON logger, you should be able to add certain [boolean] flags so you can later easily query and filter for them.

Example Flags:

```
security: true
```
```
audit: true
```
```
error: true
```

How To Log

What and How to Log

Pretty Print for local development.
JSON output for production.
Muted for automated testing.

[2021-04-10T13:05:32.712Z] ▶ notice /app/services.boot.js - Booting Services...

{"level":"notice","severity":5,"dt":"2021-04-10T13:04:03.215Z","message":"Booting Services...","context":{"runtime":{"application":"skgbot"},"source":{"file_name":"/app/services.boot.js"},"system":{"hostname":"192.168.1.74","pid":49862,"process_name":"/Users/thanpolas/.nvm/versions/node/v14.15.5/bin/node"}},"event":{}}

Pretty Print

JSON

Where and When to Log

Debugging & Error Tracing

Where and When to Log

Application Boot (OS, User, NODE_ENV).
Modules / Services Booting Up.
Node.js Error Handlers (uncaughtException, uncaught promise rejection).
Express & http Error Handlers.
All your error catch statements, if they don't bubble up.

Accounting, Audit, Security

Where and When to Log

All mutating operations.
Becomes an Audit Trail when saved on immutable store.
- Highly Secure Application require audit trails on querying as well.
Use flags to tag log types (audit, security).

Accounting, Audit, Security

Where and When to Log

Beware of PII!

(emails, IPs, names)

tokenise data

How Logality Can Help

thanpolas/logality

What is Logality?

Logality is a versatile and extensible logger for Node.JS

Logality Features

JSON and Pretty Print log messages.
Extend or alter the logging schema to fit your needs.
Built-in, over-writable serialisers.
Custom serialisers.
Middleware support.

Logality Features

Allows full manipulation of output.
Asynchronous operation.
Use in libraries and compose multiple Logality instances on the root project.
Automatically detects the module filename and path and includes them in the log.

JSON Output

Logality Features

{
    "severity": 6,
    "level": "info",
    "dt": "2018-05-18T16:25:57.815Z",
    "message": "hello world",
    "event": {},
    "context": {
        "runtime": {
            "application": "testLogality"
        },
        "source": {
          "file_name": "/test/spec/surface.test.js"
        },
        "system": {
            "hostname": "localhost",
            "pid": 36255,
            "process_name": "node ."
        }
    }
}

Automatic Detection

Pretty Print Output

Logality Features

Logality Uses the Syslog Severity Levels (RFC 5424)

Logality Serialisers

Logality Features

Serialisers are triggered by defined keys in the context object.

Each serialiser is configured to listen to a specific key.

log.info('User Logged in', {
    user: udo,
});

Built-In Serialisers

Logality Features

User Serialiser

log.info('User Logged in', { user: user })

Output

"context": {
    "user": {
        "id": 10,
        "email": "one@go.com",
    }
}

Built-In Serialisers

Logality Features

Output

"event":{
    "error":{
        "name":"Error",
        "message":"Broke",
        "backtrace": "Stack Trace...",
    }
}

Error Serialiser

log.error('User Logged in', { error: exception })

Built-In Serialisers

Logality Features

Output

"event":{
    "http_request": {
        "headers": {},
        "host": "localhost",
        "method": "GET",
        "path": "/",
        "query_string": "",
        "scheme": "http"
    }
}

Express Request Serialiser

function index (req, res) {
  log.info('Index request', { req });
}

Custom Serialisers

Logality Features

You can define your own serialisers

const mySerialisers = {
  order: function (order) {
    return {
      path: 'context.order',
      value: {
        order_id: order.id,
        sku_id: order.sku,
        total_price: order.item_price * order.quantity,
        quantity: order.quantity,
      },
    };
  },
};

log.info('New order', {order: orderItem})

Overwrite Built-In Serialisers

Logality Features

You can overwrite built-in serialisers

const mySerialisers = {
  user: function (user) {
    return {
      path: 'context.user',
      value: {
        id: user.id,
        email: user.email,
        first_name: user.first_name,
        last_name: user.last_name,
      },
    };
  },
};

log.info('User Logged In', {user: req.user})

Middleware Support

Logality Features

Middleware are invoked after serialisers.
You may add multiple middleware.
The "logContext" object is a JS Native Object, representing the entire log message.
"logContext" is Mutable.

logality.use((logContext) => {
    delete logContext.user;
});

Async Middleware Support

Logality Features

You can configure Logality for asynchronous operation

logality.use(async (logContext) => {
  await db.write(logContext);
});

logality.use(async (logContext) => {
  await slack.send(slack.format(logContext));
});

Consequently, all logging commands need async invocation:

await log.info('Something happened');

Output Manipulation

Logality Features

You can fully manipulate the master output:

const Logality = require('logality');

const logality = Logality({
  appName: 'service-something',
  prettyPrint: false,
  async: false,
  output: (logContext) => {
    const logMessage = JSON.stringify(logContext);
    process.stdout.write(logMessage);
  },
});

The "logContext" object is a JS Native Object, representing the entire log message. It is mutable.

Use in Libraries

Logality Features

Logality can safely be used in libraries!

const thirdPartyLib = require('thirdPartyLibrary');

/** ... */

const myLogality = Logality();

myLogality.pipe(thirdPartyLib.logality);

Other Logging Libraries

Comparison

	Logality	Winston	Bunyan	Pino
JSON Output	✅	✅	✅	✅
Pretty Print	✅	✅	❌	✅
Custom Log Levels	❌	✅	❌	✅
Serialisers	✅	❌	✅	✅
Middleware	✅	✅	❌	✅
Mutate JSON Schema	✅	✅	❌	❌

Comparison

	Logality	Winston	Bunyan	Pino
Output Destination	✅	✅	✅	✅
Mutate Output	✅	✅	❌	❌
Async Operation	✅	❌	❌	❌
Filename Detection	✅	❌	❌	❌
Speed Optimised	❌	❌	❌	✅
Used in Libraries	✅	❌	❌	❌

Recap

Logging is important for:
- Debugging and troubleshooting.
- Security and Audit.
- Performance monitoring.
Common Schema and appropriate logs can make all the difference.
You have many choices for a logger, choose what is appropriate for you.

Thank you

Thanos Polychronakis

@thanpolas

https://speakerdeck.com/thanpolas

Thank you

Thanos Polychronakis

@thanpolas

https://speakerdeck.com/thanpolas

Questions?

Why Logging is Important 2

By thanpolas

Why Logging is Important 2

... and how Logality can help you with that

Why Logging is Important

...and how Logality can help you with that

Hi, I am Thanos

Today's Menu

Why Logging Is Important

Logging Helps You With...

Logging Requirements

Your Environment

Consuming Logs

Consuming Logs

Common Logging Schema

What and How to Log

A Log Message Should Contain

Who Am I?

Where and What Am I?

Flags and Tagging

How To Log

Where and When to Log

Debugging & Error Tracing

Accounting, Audit, Security

Accounting, Audit, Security

Beware of PII!

tokenise data

How Logality Can Help

What is Logality?

Logality Features

Logality Features

JSON Output

Pretty Print Output

Logality Serialisers

Built-In Serialisers

Built-In Serialisers

Built-In Serialisers

Custom Serialisers

Overwrite Built-In Serialisers

Middleware Support

Async Middleware Support

Output Manipulation

Use in Libraries

Other Logging Libraries

Other Logging Libraries

Winston

Pino.js

Bunyan

Comparison

Comparison

Recap

Thank you

Why Logging is Important 2

More from thanpolas