

Marcin "Perk" Stożek
Product Manager @ Elastic
Ops background: bare metal to K8s
Migrated Fluent Bit / Fluentd / Prometheus -> OTel Collector in production
Work for a vendor, but talk is vendor neutral; 0% AI, only YAML
$ whoami




Collection

OpenTelemetry
Collector
101
OpenTelemetry Collector
Receivers
Processors
Exporters
OpenTelemetry Collector
Receivers
Processors
Exporters
Pipelines
OpenTelemetry Collector
aka otelcol

Host Metrics Receiver



Put it all together
receivers:
hostmetrics:
scrapers:
memory:
processors:
resourcedetection/detect-host-name:
detectors:
- system
system:
hostname_sources:
- os
exporters:
otlp:
endpoint: my-otlp-endpoint:4317
service:
pipelines:
metrics:
receivers:
- hostmetrics
processors:
- resourcedetection/detect-host-name
exporters:
- otlpreceivers:
hostmetrics:
scrapers:
memory:
processors:
resourcedetection/detect-host-name:
detectors:
- system
system:
hostname_sources:
- os
exporters:
otlp:
endpoint: my-otlp-endpoint:4317
service:
pipelines:
metrics:
receivers:
- hostmetrics
processors:
- resourcedetection/detect-host-name
exporters:
- otlpData Pipeline
service:
pipelines:
metrics:
receivers: [hostmetrics]
processors: [resourcedetection/detect-host-name]
exporters: [otlp]
metrics/kafka:
receivers: [kafka]
exporters: [otlp/kafka]
logs:
receivers: [filelog]
exporters: [otlp]
logs/kafka:
receivers: [kafka]
exporters: [otlp, otlp/kafka]
traces:
receivers: [otlp, kafka]
processors: [resourcedetection/detect-host-name]
exporters: [otlp, otlp/kafka]More Data Pipelines
The "What"
OTel Collector Builder
OTel Collector Distros
Third-party Distros
OTel Collector Builder
OTel Collector Distros
Third-party Distros
Core distribution
github.com/open-telemetry/opentelemetry-collector - components
Contrib distribution
https://github.com/open-telemetry/opentelemetry-collector-contrib - components
Kubernetes distribution
eBPF Profiling distribution
OTLP distribution
OTel Collector Distros

Should you use contrib?

🤷♂️
Should you use contrib?
OTel Collector Builder
OTel Collector Distros
Third-party Distros
Collectors Zoo













Collectors Zoo













Industry-wide support
Among others:
OTel Collector based, maintained by third parties.
Not validated by the community.
- ADOT (AWS)
- Alloy (Grafana)
- BDOT (Bindplane)
- DDOT (Datadog)
- EDOT (Elastic)
- NRDOT (NewRelic)
- RHOSDT (RedHat)
- SDOT (Splunk)
- Tulip (OllyGarden)
- ...
Third-party Distros
OTel Collector Builder
OTel Collector Distros
Third-party Distros
OTel Collector Builder
Lean binary
- Smaller attack surface
- Smaller footprint
- Custom components
- Self-serve
Use ocb tool for your own OTel Distro!
OTel Collector Builder
YAML manifest
GO compile
YAML manifest
dist:
name: otelcol-dev
description: Basic OTel Collector distribution for Developers
output_path: ./otelcol-dev
exporters:
- gomod:
go.opentelemetry.io/collector/exporter/debugexporter v0.165.0
- gomod:
go.opentelemetry.io/collector/exporter/otlpexporter v0.165.0
processors:
- gomod:
go.opentelemetry.io/collector/processor/batchprocessor v0.165.0
receivers:
- gomod:
go.opentelemetry.io/collector/receiver/otlpreceiver v0.165.0
providers:
- gomod:
go.opentelemetry.io/collector/confmap/provider/envprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/fileprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/httpprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.71.0
- gomod:
go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.71.0OTel Collector Builder
OTel Collector Builder
YAML manifest
GO compile
GO compile
OTel Collector Builder
$ ocb --config builder-config.yaml2026-11-18T20:17:19.001+0100 INFO internal/command.go:97 OpenTelemetry Collector Builder {"version": "v0.165.0+dirty"}
2026-11-18T20:17:19.001+0100 INFO internal/command.go:102 Using config file {"path": "builder-config.yaml"}
2026-11-18T20:17:19.002+0100 INFO builder/config.go:292 Using go {"go-executable": "/opt/homebrew/bin/go"}
2026-11-18T20:17:19.003+0100 INFO builder/main.go:110 Sources created {"path": "./otelcol-dev"}
2026-11-18T20:17:26.534+0100 INFO builder/main.go:231 Getting go modules
2026-11-18T20:17:28.888+0100 INFO builder/main.go:121 Compiling
2026-11-18T20:17:33.204+0100 INFO builder/main.go:153 Compiled {"binary": "./otelcol-dev/otelcol-dev"}
$ file otelcol-dev/otelcol-dev
otelcol-dev/otelcol-dev: Mach-O 64-bit executable arm64The "Where"
At the Gateway
At the Edge
At the Gateway
At the Edge
At the Edge
App to SDK
Sidecar
Collector
App to file
Daemonset Collector
Push
Pull
Collection where the workload is,
e.g. using Host agent Collector on VM or bare metal
At the Edge
- Redaction: PII
- Enrichment
- K8s Attributes
- DNS lookup
- Stateless sampling
- e.g. errors only
Main goals:
-
protect the app
-
zero trust
-
isolation
At the Gateway
At the Edge
data flood
At the Gateway
SDK
Collector
Sidecar
Gateway





At the Gateway
SDK
Collector
Sidecar
Gateway









- Complex routing
- Stateful sampling
- e.g. tail based
- Rate limiting
- Multi-backend fan-out
At the Gateway
Main goals:
-
protect the data
-
secure boundary (telemetry firewall)
Gateway security
The secure boundary
- Centralized API keys management
- Data masking and PII scrubbing
- Traffic validation
Gateway resilience
Protect the data
- Persistent queue
- Retry and backoff strategy
- Stateful load balancing
Edge security
Zero trust and isolation
- Network isolation
- Least privilege execution
- No vendor secrets at the edge
Edge resilience
Protect the app
- Prevent OOM kills
- Fail fast (drop over block)
- Strict resource limits
The "How"
Day 2 operations
# manual drift…
apiVersion: v1
kind: ConfigMap
metadata:
name: otel-gateway
data:
config.yaml: |
processors:
tail_sampling:
policies:
- - name: keep-errors
+ - name: keep-errors-and-checkout
type: string_attribute
...$ ssh ops@edge-482.mgmt
Connection timed out
$ ssh ops@edge-483.mgmt
Connection timed out
$ for h in $(cat hosts.txt); do
ssh $h 'sudo systemctl restart otelcol'
done
ssh: connect to host edge-511 port 22:
Operation timed out
# still 600 nodes to go…Manual ConfigMap surgery, hop-by-hop SSH, configuration drift, partial rollouts... What can go wrong?
OTel fleet mgmt evolution
Static config
GitOps / Helm
Operators
OpAMP
?

OTel fleet mgmt evolution
Static config
GitOps / Helm
Operators
OpAMP
?
OTel fleet mgmt evolution
Static config
GitOps / Helm
Operators
OpAMP
✓
Why OpAMP?
- World is not 100% K8s
- Multi-cluster sprawl
- Actual vs desired state monitoring
- Can manage SDKs too!
Meet OpAMP (beta)
Components
- Specification (beta)
- Go reference impl (v0.x)
- OpAMP Extension (alpha)
- OpAMP Supervisor (alpha)
- OpAMP Gateway (alpha, not yet upstream)
OpAMP in action
OpAMP backend
Telemetry backend
OpenTelemetry Collector
OpAMP Supervisor
OTLP data
OpAMP monitoring
Config
Config
Recap
Control-plane early
Start with the Operator if you can; design headroom for OpAMP when the fleet spans the platform boundary.
Topology
Adopt the edge-to-gateway pattern as the foundation for scale and clear trust boundaries.
Binary on purpose
Prefer OCB or a curated distribution (upstream or vendor) over shipping full contrib to every production node.
From Edge to Gateway
Managed Data Pipelines
Some more #OpenTelemetry
Community
Thank you!
Marcin "Perk" Stożek
@marcinstozek / perk.pl

From Edge to Gateway - architecting your OpenTelemetry Collector deployment
By Marcin Stożek
From Edge to Gateway - architecting your OpenTelemetry Collector deployment
- 387