Logs are not enough

...or why traces are worth the setup.

@radekmie

About me

@radekmie

Author of uniforms.

Everything is at radekmie.dev.

Core contributor of Meteor.js.

Head of Engineering at aleno.

@radekmie

What are logs?

A log is a timestamped text record, either structured (recommended) or unstructured, with optional metadata. [...]


 

 

A log is a timestamped text record,


Most programming languages have built-in logging capabilities or well-known, widely used logging libraries.
A log is a timestamped text record,




 

~ OpenTelemetry docs

@radekmie

How to work with logs?

async function installPayments(c: Config, r: Id) {
  logger.info('Install started.');
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info('Install finished.');
}

















async function createStripeAccount(c: Config) {
  const response = await fetch(...);
  if (!response.ok)
    throw new Error('StripeError');
  const result = await response.json();
  return result.id;
}
@radekmie

How to work with logs?

async function installPayments(c: Config, r: Id) {
  logger.info('Install started.');
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info('Install finished.');
}

async function createStripeAccount(c: Config) {
  const response = await fetch(...);
  if (!response.ok)
    throw new Error('StripeError');
  const result = await response.json();
  return result.id;
}

Given these logs, answer these questions:

  1. # of installations last week.
  2. Average duration.
  3. % of fails.
@radekmie

How to work with logs?

async function installPayments(c: Config, r: Id) {
  logger.info('Install started.');
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info('Install finished.');
}

async function createStripeAccount(c: Config) {
  const response = await fetch(...);
  if (!response.ok)
    throw new Error('StripeError');
  const result = await response.json();
  return result.id;
}

Given these logs, answer these questions:

  1. When was the last attempt from X?
  2. Did they succeed?
  3. What was the Stripe error?
@radekmie

We need context!

async function installPayments(c: Config, r: Id) {
  logger.info(`Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`Install finished for ${r}.`);
}

async function createStripeAccount(c: Config) {
  const response = await fetch(...);
  if (!response.ok)
    throw new Error(`StripeError${response.status}`);
  const result = await response.json();
  return result.id;
}

Unstructured!

@radekmie

Actually, lot's of it...

async function checkIfChargeAlreadyExists(...) {
  const existingTransactions =
    await transactions.search(spaceId, { ... });

  if (existingTransactions.body.length > 0) {
    logger.warn(
      'Duplicate charge transaction detected.',
      { referenceId, restaurantId, spaceId, token },
    );
    throw Errors.DUPLICATE_CHARGE_DETECTED;
  }
}

Structured!

@radekmie

CloudWatch Logs Insights

filter @message like /error/
| parse @message /"restaurantId":"(?<id>[^"]+)"/
| stats count(*) as total by message, id
| sort total desc

Lots of operators, including pattern detection.

Documentation.

@radekmie

Can I somehow link logs?

async function installPayments(c: Config, r: Id) {
  logger.info(`Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`Install finished for ${r}.`);
}
[2026-06-11 18:25:10] Install started for X.

Which one is which?


[2026-06-11 18:25:11] Install started for X.




[2026-06-11 18:25:19] Install finished for X.




[2026-06-11 18:25:20] Install finished for X.
@radekmie

Execution ID

async function installPayments(c: Config, r: Id) {
  logger.info(`Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`Install finished for ${r}.`);
}

@radekmie

Execution ID

async function installPayments(c: Config, r: Id) {

  logger.info(`Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`Install finished for ${r}.`);
}

@radekmie

Execution ID

async function installPayments(c: Config, r: Id) {
  const x = getRandomExecutionId();
  logger.info(`Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`Install finished for ${r}.`);
}

@radekmie

Execution ID

async function installPayments(c: Config, r: Id) {
  const x = getRandomExecutionId();
  logger.info(       `Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(       `Install finished for ${r}.`);
}

@radekmie

Execution ID

async function installPayments(c: Config, r: Id) {
  const x = getRandomExecutionId();
  logger.info(`(${x}) Install started for ${r}.`);
  if (await database.getById(r))
    throw new Error('PaymentsAlreadyInstalled');
  const account = await createStripeAccount(c);
  await database.insert(r, account);
  logger.info(`(${x}) Install finished for ${r}.`);
}

[2026-06-11 18:25:10] (A) Install started for X.




[2026-06-11 18:25:11] (B) Install started for X.




[2026-06-11 18:25:19] (B) Install finished for X.




[2026-06-11 18:25:20] (A) Install finished for X.

AWS Lambda does it: see RequestId.

@radekmie

Execution ID

Every "operation" could generate its own ID and add it to all of its logs.

Nothing is stopping us from passing IDs down to child functions, so they could correlate with parent's logs...

...even if they are executed on a different machine. We just need to send the ID.

Congrats, we just invented tracing!

@radekmie

What is a trace?

Traces give us the big picture of what happens when a request is made to an application. Whether your application is a monolith with a single database or a sophisticated mesh of services, traces are essential to understanding the full “path” a request takes in your application.
Traces give us the big picture of what happens when a request is made to an application. Whether your application is a monolith with a single database or a sophisticated mesh of services, traces are essential to understanding the full “path” a request takes in your application.
Traces give us the big picture of what happens when a request is made to an application. Whether your application is a monolith with a single database or a sophisticated mesh of services, traces are essential to understanding the full “path” a request takes in your application.

~ OpenTelemetry docs

@radekmie

How to work with traces?

async function installPayments(c: Config, r: Id) {
  await trace('installPayments', { r }, async () => {
    if (await database.getById(r))
      throw new Error('PaymentsAlreadyInstalled');
    const account = await createStripeAccount(c);
    await database.insert(r, account);
  });
}

















function createStripeAccount(c: Config) {
  return trace('createStripeAccount', async () => {
    const response = await fetch(...);
    if (!response.ok)
      throw new Error('StripeError');
    const result = await response.json();
    return result.id;
  });
}

trace() calls can be nested!

@radekmie

How does it look?

@radekmie

How does it look?

@radekmie

What about the costs?

Provider Read Write Storage
Dynatrace $0.0035/GB $0.20/GB $0.21/GB
Grafana Labs $0.05/GB $0.40/GB $0.10/GB
Signoz (included) $0.40/GB (included)

Self hosting is a viable option, too! Just make sure to account for the time spent on maintaining it.

@radekmie

What else is there!?

  1. Span kinds, links, and events.
  2. Automatic metrics generation.
  3. Service graph.
  4. Logs correlation.
  5. TraceQL.

Read my On Distributed Tracing for more technical details and even more links.

@radekmie

Questions?

Logs are not enough

By Radosław Miernik

Logs are not enough

Meet.js Wrocław, 2026-06-11.

  • 3

More from Radosław Miernik