It is interesting why Airbnb uses vmagent for streaming aggregation and didn't switch from Mimir to VictoriaMetrics. This could save them a lot of costs on infrastructure and operations, like in cases of Roblox, Spotify, Grammarly and others - https://docs.victoriametrics.com/victoriametrics/casestudies...
Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
21–26 of 26 posts
Re: Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
#22the zero injection fix for sparse counters is the most underrated part of this writeup -- injecting a synthetic zero on first flush to anchor the cumulative baseline is actaully a pretty elegant solution to a problem that bites almost every team migrating from delta-based systems to prometheus, and the fact that they centralized it in the aggregation tier rather than pushing the fix to every instrumentation callsite…
There is another approach for solving this issue - to use increase_pure() function from MetricsQL - https://docs.victoriametrics.com/metricsql/#increase_pure . Of course, you need to switch to VictoriaMetrics, since Mimir doesn't support this function.
Re: Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
#23the zero injection fix for sparse counters is the most underrated part of this writeup -- injecting a synthetic zero on first flush to anchor the cumulative baseline is actaully a pretty elegant solution to a problem that bites almost every team migrating from delta-based systems to prometheus, and the fact that they centralized it in the aggregation tier rather than pushing the fix to every instrumentation callsite…
I was under impression that problem of zero injection was solved with Start Timestamp from OpenMetrics 2.0 spec - see https://prometheus.io/docs/specs/om/open_metrics_spec_2_0/#s...
Re: Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
#24It is interesting why Airbnb uses vmagent for streaming aggregation and didn't switch from Mimir to VictoriaMetrics. This could save them a lot of costs on infrastructure and operations, like in cases of Roblox, Spotify, Grammarly and others - https://docs.victoriametrics.com/victoriametrics/casestudies...
Could you share a little more about your involvement with VictoriaMetrics? A good faith disclosure goes a long way.
Re: Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
#25It is interesting why Airbnb uses vmagent for streaming aggregation and didn't switch from Mimir to VictoriaMetrics. This could save them a lot of costs on infrastructure and operations, like in cases of Roblox, Spotify, Grammarly and others - https://docs.victoriametrics.com/victoriametrics/casestudies...
Could you share a little more about your involvement with VictoriaMetrics? A good faith disclosure goes a long way.
I'm core developer at VictoriaMetrics. This information is one click away - just click my name here.
Re: Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus
#26Interesting choice to go with Prometheus directly, especially when other TSDBs have "native" support for OTLP ingestion support.