Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Managed Service for Prometheus

aws.amazon.com

Post reply on HN
Print view View original

31–40 of 77 posts

Re: Amazon Managed Service for Prometheus

#31

Post by RocketSyntax »

john_moscow wrote: post #29
latchkey wrote: post #26

I just went through the "process" of installing Grafana, Loki, Promtail and Prometheus on an ubuntu box and it is almost like the company behind all of this has gone out of the their way to make it hard. It isn't really _that_ difficult to get set up, but it also isn't 'apt install' easy (you really want me to create my own startup scripts?) and required me to build my own documentation on how I installed everything.

It's almost like the company behind it wants to see some profit after pouring millions of dollars into developing these tools. Except, in 2020 you cannot just have a closed-source easy-to-use documented and supported product with a license fee. Not in the server market, at least. Everything must be free and open-source, and you are expected to make money by offering a hosted service. Except, good luck competing with…

It's extremely worrisome. The incentive to spend your early mornings, nights, and weekends building something awesome to free yourself from corporate life is fading away. They need to institute some kind of royalty program or at least dedicate engineers to helping maintain the projects they make into services.

Almost have to change gears and get into a scientific field that isn't computer science.

Re: Amazon Managed Service for Prometheus

#32

Post by rfratto »

latchkey wrote: post #26

I just went through the "process" of installing Grafana, Loki, Promtail and Prometheus on an ubuntu box and it is almost like the company behind all of this has gone out of the their way to make it hard. It isn't really _that_ difficult to get set up, but it also isn't 'apt install' easy (you really want me to create my own startup scripts?) and required me to build my own documentation on how I installed everything.

One of the Loki maintainers here (though I mostly work on other stuff now). I promise it's not difficult on purpose.

We've put a lot of effort into optimizing the Kubernetes experience that non-containerized installations haven't been getting as much attention. We'd be thrilled to have system packages for Loki that also set it up as a service, it's just not something we've been able to spend time doing ourselves yet.

Re: Amazon Managed Service for Prometheus

#33

Post by 0xbadcafebee »

latchkey wrote: post #26

I just went through the "process" of installing Grafana, Loki, Promtail and Prometheus on an ubuntu box and it is almost like the company behind all of this has gone out of the their way to make it hard. It isn't really _that_ difficult to get set up, but it also isn't 'apt install' easy (you really want me to create my own startup scripts?) and required me to build my own documentation on how I installed everything.

All of those who have spent their free time contributing to Linux distributions are why 'apt install' is easy. You can contribute too.

Re: Amazon Managed Service for Prometheus

#35

Post by bboreham »

edoceo wrote: post #5
pickledish wrote: post #2

14 cents per "query processing minute" sounds like it could add up very fast. Prom queries can get somewhat complex and it's not rare at all IME to have a dashboard making several multi-second queries per load (whether that falls into "you're using Prometheus wrong" being a separate discussion of course) Edit: The example from their pricing page: > We will assume you have 1 end user monitoring a dashboard for an aver…

Now do six dashboards, 10 widgets each, multiple viewers, 18h/day and one slowish query on each dashboard. Seems like we get to hundred+ pretty quick

Caching means that multiple viewers cost very little extra.

(I am a Cortex maintainer)

Re: Amazon Managed Service for Prometheus

#36

Post by Thaxll »

pram wrote: post #28
Thaxll wrote: post #21

Earlier quoted context omitted.

Prometheus is not easy to run at scale on the storage side.

This is all relative but I don't personally think so. Not on EC2+EBS, anyway. Certainly not as difficult as running/scaling an ES or Kafka cluster.

It's a completely different problem because by default Prometheus does not shard anything so you're bound to a single instance, where ES and Kafka are cluster based.

Re: Amazon Managed Service for Prometheus

#37

Post by slyall »

The pricing just for the ingest seems way off. $0.002 for 10,000 metrics might not seem like much by even a simple node_exporter will grab 700 metrics every 15 seconds.

Thats $24/month just to ingest the cpu/ram/diskspace data from each server. Plus storage and query costs.

At work I have a single r4.xlarge instance handling 1.3 million metrics every 15 seconds. Storage is not clustered but cost is only $500/month. It would cost me $45k/month just for the ingest with the new managed service.

Re: Amazon Managed Service for Prometheus

#38

Post by latchkey »

0xbadcafebee wrote: post #33
latchkey wrote: post #26

I just went through the "process" of installing Grafana, Loki, Promtail and Prometheus on an ubuntu box and it is almost like the company behind all of this has gone out of the their way to make it hard. It isn't really _that_ difficult to get set up, but it also isn't 'apt install' easy (you really want me to create my own startup scripts?) and required me to build my own documentation on how I installed everything.

All of those who have spent their free time contributing to Linux distributions are why 'apt install' is easy. You can contribute too.

As the co-founder of Apache Java and a 20+ year member of the ASF, creator and contributor to hundreds of projects over the years, I think I've contributed enough of my time to OSS. I'm more than happy to let the new kids jump in. Thanks for the 'advice'.

Re: Amazon Managed Service for Prometheus

#39

Post by latchkey »

rfratto wrote: post #32
latchkey wrote: post #26

I just went through the "process" of installing Grafana, Loki, Promtail and Prometheus on an ubuntu box and it is almost like the company behind all of this has gone out of the their way to make it hard. It isn't really _that_ difficult to get set up, but it also isn't 'apt install' easy (you really want me to create my own startup scripts?) and required me to build my own documentation on how I installed everything.

One of the Loki maintainers here (though I mostly work on other stuff now). I promise it's not difficult on purpose. We've put a lot of effort into optimizing the Kubernetes experience that non-containerized installations haven't been getting as much attention. We'd be thrilled to have system packages for Loki that also set it up as a service, it's just not something we've been able to spend time doing ourselves yet.

It isn't just loki, but the whole stack. Grafana is the only project mentioned that has a debian installer.

The expectation that someone doing greenfield development is going to jump into k8s just to use the software is kind of weird.

Re: Amazon Managed Service for Prometheus

#40

Post by mchusma »

slyall wrote: post #37

The pricing just for the ingest seems way off. $0.002 for 10,000 metrics might not seem like much by even a simple node_exporter will grab 700 metrics every 15 seconds. Thats $24/month just to ingest the cpu/ram/diskspace data from each server. Plus storage and query costs. At work I have a single r4.xlarge instance handling 1.3 million metrics every 15 seconds. Storage is not clustered but cost is only $500/month. I…

Their pricing for these managed services used to be "no brainer" (something like the cost of compute only, or maybe a <30% upcharge). Managed airflow was similarly very expensive (maybe 3x the cost). Just not worth it. Bummer.
Post reply on HN