Marcin "Perk" Stożek
Product Manager @ Elastic
Ops background: bare metal to K8s
Migrated Fluent Bit / Fluentd / Prometheus -> OTel Collector in production
Work for a vendor, but talk is vendor neutral; 0% AI, only YAML
receivers:
hostmetrics:
scrapers:
memory:
processors:
resourcedetection/detect-host-name:
detectors:
- system
system:
hostname_sources:
- os
exporters:
otlp:
endpoint: my-otlp-endpoint:4317
service:
pipelines:
metrics:
receivers:
- hostmetrics
processors:
- resourcedetection/detect-host-name
exporters:
- otlpreceivers:
hostmetrics:
scrapers:
memory:
processors:
resourcedetection/detect-host-name:
detectors:
- system
system:
hostname_sources:
- os
exporters:
otlp:
endpoint: my-otlp-endpoint:4317
service:
pipelines:
metrics:
receivers:
- hostmetrics
processors:
- resourcedetection/detect-host-name
exporters:
- otlpservice:
pipelines:
metrics:
receivers: [hostmetrics]
processors: [resourcedetection/detect-host-name]
exporters: [otlp]
metrics/kafka:
receivers: [kafka]
exporters: [otlp/kafka]
logs:
receivers: [filelog]
exporters: [otlp]
logs/kafka:
receivers: [kafka]
exporters: [otlp, otlp/kafka]
traces:
receivers: [otlp, kafka]
processors: [resourcedetection/detect-host-name]
exporters: [otlp, otlp/kafka]Core distribution
github.com/open-telemetry/opentelemetry-collector - components
Contrib distribution
https://github.com/open-telemetry/opentelemetry-collector-contrib - components
Kubernetes distribution
eBPF Profiling distribution
OTLP distribution
Among others:
OTel Collector based, maintained by third parties.
Not validated by the community.
Lean binary
Use ocb tool for your own OTel Distro!
YAML manifest
GO compile
YAML manifest
dist:
name: otelcol-dev
description: Basic OTel Collector distribution for Developers
output_path: ./otelcol-dev
exporters:
- gomod:
go.opentelemetry.io/collector/exporter/debugexporter v0.165.0
- gomod:
go.opentelemetry.io/collector/exporter/otlpexporter v0.165.0
processors:
- gomod:
go.opentelemetry.io/collector/processor/batchprocessor v0.165.0
receivers:
- gomod:
go.opentelemetry.io/collector/receiver/otlpreceiver v0.165.0
providers:
- gomod:
go.opentelemetry.io/collector/confmap/provider/envprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/fileprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/httpprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.71.0YAML manifest
GO compile
GO compile
$ ocb --config builder-config.yaml2026-11-18T20:17:19.001+0100 INFO internal/command.go:97 OpenTelemetry Collector Builder {"version": "v0.165.0+dirty"}
2026-11-18T20:17:19.001+0100 INFO internal/command.go:102 Using config file {"path": "builder-config.yaml"}
2026-11-18T20:17:19.002+0100 INFO builder/config.go:292 Using go {"go-executable": "/opt/homebrew/bin/go"}
2026-11-18T20:17:19.003+0100 INFO builder/main.go:110 Sources created {"path": "./otelcol-dev"}
2026-11-18T20:17:26.534+0100 INFO builder/main.go:231 Getting go modules
2026-11-18T20:17:28.888+0100 INFO builder/main.go:121 Compiling
2026-11-18T20:17:33.204+0100 INFO builder/main.go:153 Compiled {"binary": "./otelcol-dev/otelcol-dev"}
$ file otelcol-dev/otelcol-dev
otelcol-dev/otelcol-dev: Mach-O 64-bit executable arm64The secure boundary
Protect the data
Zero trust and isolation
Protect the app
# manual drift…
apiVersion: v1
kind: ConfigMap
metadata:
name: otel-gateway
data:
config.yaml: |
processors:
tail_sampling:
policies:
- - name: keep-errors
+ - name: keep-errors-and-checkout
type: string_attribute
...$ ssh ops@edge-482.mgmt
Connection timed out
$ ssh ops@edge-483.mgmt
Connection timed out
$ for h in $(cat hosts.txt); do
ssh $h 'sudo systemctl restart otelcol'
done
ssh: connect to host edge-511 port 22:
Operation timed out
# still 600 nodes to go…Manual ConfigMap surgery, hop-by-hop SSH, configuration drift, partial rollouts... What can go wrong?
OpAMP backend
Telemetry backend
OpenTelemetry Collector
OpAMP Supervisor
OTLP data
OpAMP monitoring
Config
Config
Start with the Operator if you can; design headroom for OpAMP when the fleet spans the platform boundary.
Adopt the edge-to-gateway pattern as the foundation for scale and clear trust boundaries.
Prefer OCB or a curated distribution (upstream or vendor) over shipping full contrib to every production node.
Marcin "Perk" Stożek
@marcinstozek / perk.pl