Prometheus reaches 1.0 • Hacker News Board

Re: Prometheus reaches 1.0

#11

Post by andmarios » Mon, Jul 18, 2016, 4:46 PM UTC

Congrats! Prometheus + grafana are a killer duet! Easy to setup and use. Same goes for node_exporter. I only wish more JVM apps included a configuration file for the jmx_exporter and an easier to setup nginx_exporter. :)

Incidentally yesterday I created a simple exporter for (linux) coretemp, hddtemp and NUT's upsc: https://github.com/andmarios/sensor_exporter

Re: Prometheus reaches 1.0

#12

Post by jrv » Mon, Jul 18, 2016, 4:48 PM UTC

j_s wrote: post #9

What is the best GitHub issue to follow to find out when Prometheus supports events older than the configurable 5 minute limit? I'm blocked because a network partition can mean no stats. Edit: Maybe https://github.com/prometheus/prometheus/issues/398 ? Also, limit is configurable.

That would be the correct issue about the staleness limit, yes. Note that Prometheus does not track individual events, but only numerical time series and their current and historical values. What's the exact use case you're stuck on? I guess you are using the pushgateway with client-side timestamps?

Re: Prometheus reaches 1.0

#13

Post by xvf33 » Mon, Jul 18, 2016, 4:59 PM UTC

I my have misread the documentation but there seems to be no way to get the same output as the highestAverage, highestCurrent, or highestCurrent functions from graphite.

Not sure how to filter through say 300 servers and select out the top 10 for a particular time span. I would think that it would be a common need but I guess I'm missing something?

Re: Prometheus reaches 1.0

#14

Post by _lbaq » Mon, Jul 18, 2016, 5:04 PM UTC

jrv wrote: post #2

Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!

What is the typical user/customer of this system ? I've found it very hard to get customers to spend time configuring a motoring system, let alone writing any scripts to gather data.

Re: Prometheus reaches 1.0

#15

Post by bbrazil » Mon, Jul 18, 2016, 5:06 PM UTC

xvf33 wrote: post #13

I my have misread the documentation but there seems to be no way to get the same output as the highestAverage, highestCurrent, or highestCurrent functions from graphite. Not sure how to filter through say 300 servers and select out the top 10 for a particular time span. I would think that it would be a common need but I guess I'm missing something?

We've a slightly different computational model, so that takes two passes.

You can calculate the highest value now (topk) or the highest averages (topk+avg_over_time) and now that you know which timeseries you want, graph those. I believe this is doable in Grafana.

Re: Prometheus reaches 1.0

#16

Post by jrv » Mon, Jul 18, 2016, 5:10 PM UTC

xvf33 wrote: post #13

I my have misread the documentation but there seems to be no way to get the same output as the highestAverage, highestCurrent, or highestCurrent functions from graphite. Not sure how to filter through say 300 servers and select out the top 10 for a particular time span. I would think that it would be a common need but I guess I'm missing something?

Prometheus range queries work a bit differently, so this is not 100% reproducible in a graph query, but a similar thing is possible. A given PromQL expression is evaluated at every resolution step along the graph and doesn't have context about what the "graph range" is. At every evaluation point, it can still look back over a given time window, but that's more of a sliding window approach then and independent of what the visible graph time range is.

For example, instead of highestCurrent, you could do something like:

topk(3, my_metric)

This would at every point along the graph select the current top 3 series that have the my_metric metric name.

Or if you want to average each series over the last 10 minutes at every point in the graph before selecting the top 3:

topk(3, avg_over_time(my_metric[10m]))

Note that due to the reasons mentioned above, topk() here does not select whatever line has the largest area under the entire visible graph range, but whatever is at the top at each given resolution step. So you may actually get more than 3 series in your graph, but only 3 at a time at any given X.

There's also an issue asking about this, but we're not sure if that is fundamentally compatible with Prometheus's query execution model without major changes: https://github.com/prometheus/prometheus/issues/586

Re: Prometheus reaches 1.0

#17

Post by bbrazil » Mon, Jul 18, 2016, 5:12 PM UTC

_lbaq wrote: post #14

jrv wrote: post #2

Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!

What is the typical user/customer of this system ? I've found it very hard to get customers to spend time configuring a motoring system, let alone writing any scripts to gather data.

I'm not sure there is a typical user. We've got everything from single-sysadmin small companies to Fortune 500s to companies looking to integrate it into their products.

https://prometheus.io/blog/2016/03/23/interview-with-life360... and https://prometheus.io/blog/2016/05/01/interview-with-showmax... look at two of our users.

Re: Prometheus reaches 1.0

#18

Post by [deleted] » Mon, Jul 18, 2016, 5:12 PM UTC

_lbaq wrote: post #14

jrv wrote: post #2

Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!

What is the typical user/customer of this system ? I've found it very hard to get customers to spend time configuring a motoring system, let alone writing any scripts to gather data.

[deleted]

Re: Prometheus reaches 1.0

#19

Post by jrv » Mon, Jul 18, 2016, 5:14 PM UTC

_lbaq wrote: post #14

jrv wrote: post #2

Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!

What is the typical user/customer of this system ? I've found it very hard to get customers to spend time configuring a motoring system, let alone writing any scripts to gather data.

Yeah, it goes from people putting Prometheus on their Raspberry PI all the way to companies like DigitalOcean monitoring millions of machines with Prometheus (https://promcon.io/talks/scaling_to_a_million_machines_with_...).

A lot of open-source projects (Kubernetes, Etcd, ...) are also exposing native Prometheus metrics now, making it easier to integrate with those.

EDIT: also check the PromCon schedule for other user companies giving talks: https://promcon.io/schedule/

And the sponsors are also users: https://promcon.io/#our-sponsors

Re: Prometheus reaches 1.0

#20

Post by lyonlim » Mon, Jul 18, 2016, 5:21 PM UTC

jrv wrote: post #2

Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!

Is it even right to try to compare this with New Relic? Prometheus looks very interesting and I'm trying to figure out if this is something useful to us. Thanks!