Prometheus 2.0

#1

Post by bbrazil » Wed, Nov 08, 2017, 8:24 AM UTC

Prometheus 2.0
prometheus.io

Re: Prometheus 2.0

#2

Post by SkyRocknRoll » Wed, Nov 08, 2017, 9:24 AM UTC

Does this storage engine onpar with influxdb ?

Re: Prometheus 2.0

#3

Post by will-not-smith » Wed, Nov 08, 2017, 9:38 AM UTC

SkyRocknRoll wrote: post #2

Does this storage engine onpar with influxdb ?

Nope, not on par.

While there is no direct comparison to InfluxDB in the article, look at the disk usage and (more importantly) disk I/O utilization, which is indistinguishable from zero on the graph, whereas before it was at 20-30% with the same load.

So not on par. Much, much better.

(Of course, Prometheus and InfluxDB are not perfect substitutes for one another, so there's much more to look at than just storage engine performance.)

Re: Prometheus 2.0

#4

Post by bbrazil » Wed, Nov 08, 2017, 9:51 AM UTC

will-not-smith wrote: post #3

SkyRocknRoll wrote: post #2

Does this storage engine onpar with influxdb ?

Nope, not on par. While there is no direct comparison to InfluxDB in the article, look at the disk usage and (more importantly) disk I/O utilization, which is indistinguishable from zero on the graph, whereas before it was at 20-30% with the same load. So not on par. Much, much better. (Of course, Prometheus and InfluxDB are not perfect substitutes for one another, so there's much more to look at than just storage en…

The third version of the InfluxDB database is basically the same core design as Prometheus's third database version which 2.0 uses. I'd expect performance to be broadly similar, maybe a little better with Prometheus as we can take advantage of the characteristics of our domain.

We do have a comparison to Influx on our website: https://prometheus.io/docs/introduction/comparison/#promethe... Which is the right choice really depends on the use case. If you're doing metrics-based operational monitoring, Prometheus is generally best.

Disclaimer: Prometheus developer.

Re: Prometheus 2.0

#5

Post by frik » Wed, Nov 08, 2017, 10:52 AM UTC

Does Prometheus 2 still use the unmaintained charting library "flot charts" ?

http://www.flotcharts.org/

No commits since 2014. Unfortunately, because it's a great chart library (actually probably the best free one), only the reliance on JQuery isn't that fashionable.

Edit: oh, it was Grafana and Kibana that used Flot charts (at least when I tried it two years ago, maybe things have changed) quick Google (2014): https://github.com/grafana/grafana/issues/222

Re: Prometheus 2.0

#6

Post by jrv » Wed, Nov 08, 2017, 10:54 AM UTC

frik wrote: post #5

Does Prometheus 2 still use the unmaintained charting library "flot charts" ? http://www.flotcharts.org/ No commits since 2014. Unfortunately, because it's a great chart library (actually probably the best free one), only the reliance on JQuery isn't that fashionable. Edit: oh, it was Grafana and Kibana that used Flot charts (at least when I tried it two years ago, maybe things have changed) quick Google (2014): http…

No, we never used flot charts. We're using Rickshaw since the beginning: http://code.shutterstock.com/rickshaw/

It was unmaintained for a while, but now it seems there've been commits in the recent past again. However, for real dashboarding (not just simple ad-hoc queries), we recommend using Grafana anyway.

Re: Prometheus 2.0

#7

Post by sz4kerto » Wed, Nov 08, 2017, 11:30 AM UTC

No good migration story for existing data, that'll hurt us quite a lot. :(

Re: Prometheus 2.0

#8

Post by jrv » Wed, Nov 08, 2017, 11:39 AM UTC

sz4kerto wrote: post #7

No good migration story for existing data, that'll hurt us quite a lot. :(

Yeah, as Prometheus's local storage is meant more as a transient / non-durable metrics store, the only current way to migrate while simultaneously accessing old and new data is to run both the old and new servers and have the new one read old data from the other one via the remote-read integration.

Someone could write a tool to do a full migration of the old storage format to the new one, but the formats are completely different and at least in the naive version of such tooling, that would have to happen offline and take a very long time to run for large storages.

EDIT: If you would like to fund development of such a tool, let us know :)

Re: Prometheus 2.0

#9

Post by Dowwie » Wed, Nov 08, 2017, 11:53 AM UTC

The TimescaleDB crew built a Prometheus-PG adapter [1]. TimescaleDB, and consequently Postgres, is an option I am exploring. Does a developer realize any benefits of 2.0 if the Prometheus database isn't used as a backend?

[1] https://github.com/timescale/prometheus-postgresql-adapter

Re: Prometheus 2.0

#10

Post by jrv » Wed, Nov 08, 2017, 12:05 PM UTC

Dowwie wrote: post #9

The TimescaleDB crew built a Prometheus-PG adapter [1]. TimescaleDB, and consequently Postgres, is an option I am exploring. Does a developer realize any benefits of 2.0 if the Prometheus database isn't used as a backend? [1] https://github.com/timescale/prometheus-postgresql-adapter

I am not too familiar with the PG-TimescaleDB adapter, but the storage engine rewrite in Prometheus was definitely the most important improvement in Prometheus 2.0.

There are other changes than the storage though that are outlined in the migration guide: https://prometheus.io/docs/prometheus/latest/migration/

* Minor PromQL changes (mainly removals of deprecated features)

* Staleness handling (not sure if the TimescaleDB storage handles this correctly, as it relies on storing a special NaN value bit-wise correctly for marking series stale)

* New rule file format is now YAML and supports per-group evaluation intervals and serialization of rule executions within one group (in case of data dependencies)

* Other minor bits