We’re excited to share that Streamfold has entered into an agreement to join the team at Cursor!

TLDR;​

• Efficiency at scale matters: Small reductions in resource usage can have massive implications on cost and efficiency
• Benchmarking OTel + ClickHouse: We construct a pipeline to benchmark streaming trace data into ClickHouse
• Rotel scales 4x: We explore how we scaled from 137K trace spans/sec per-core with the OTel Collector to 462K trace spans/sec per-core with Rotel, detailing several key performance improvements
• Tools and resources: Find the tools we used to drive our benchmark evaluation

Observability shouldn't be a resource hog, that’s why at the end of last year, we started an ambitious new project – Rotel. We’ve been excited to see the rise of OpenTelemetry as a vendor neutral standard for telemetry and it's inspired us to apply our systems experience to develop a high-performance, resource efficient data plane for collecting OpenTelemetry data.

Rotel is a new open-source alternative for collecting OpenTelemetry data and is engineered to be an efficient, high-performance solution for receiving, processing, and exporting OpenTelemetry data. Rotel runs as a standalone process and consumes telemetry from external processes or other collection agents. It consumes 75% less memory and 50% less CPU in benchmarks, so it is particularly well-suited for environments where resource optimization is paramount.

Today we’re excited to announce Rotel and we welcome the community to build with us.