Hoe can I hide all Amazon and Google news on HN ? Do you know an alternative of HN without big tech lobby? Thanks.
I'm interested in this too, more I see aws dominating every aspect of our life, more depressed I become.
Amazon Managed Service for Prometheus
61–70 of 77 posts
Re: Amazon Managed Service for Prometheus
#62The pricing just for the ingest seems way off. $0.002 for 10,000 metrics might not seem like much by even a simple node_exporter will grab 700 metrics every 15 seconds. Thats $24/month just to ingest the cpu/ram/diskspace data from each server. Plus storage and query costs. At work I have a single r4.xlarge instance handling 1.3 million metrics every 15 seconds. Storage is not clustered but cost is only $500/month. I…
Pricing makes sense for enterprises. Considering that you may need a team (or a part of one) to maintain a self-hosted cluster at possibly 0.995 reliability, do upgrades, manage devs, run all the mandatory security scans, justify why some enterprise scan tool throwing errors isn't an issue, etc. Oh also justify why you need the manpower to do it, at which point your VP will tell you to just use the managed service.
Re: Amazon Managed Service for Prometheus
#63Earlier quoted context omitted.
Does it put any limits on cardinality of metrics? Grafana cloud's offering was absolutely awful for my use cases. They charge per-series so if you have metrics with a "pod=..." label your prices go through the roof.
Every managed metrics system will put a limit on cardinality because all mainstream available metrics systems cost more per cardinality to query and store. If they don’t limit that you can assume you or some other customer is going to use up the clusters resources and cause an outage. Like most metrics systems, under the covers in Prometheus each unique combination of dimensions is the same as a new metric line.
[deleted]
Re: Amazon Managed Service for Prometheus
#64Earlier quoted context omitted.
Pricing makes sense if you consider how Amazon operates at this point. You put basically a MVP product out there with abnormal pricing. Your enterprise customers that are drowning in money can start using it and using that money you can grow your org by hiring more engineers. At this point you start working on adding new features and do cost optimization. Since your whole architecture was designed based on "we have t…
And let's be honest. We all know a company or two that would throw _way_ more than 45k/year at a global metrics solution to handle that volume, and still wind up with a flaming scrap heap. And a promotion or two.
uhh, the GP said $45k/month, so that's $540k/year.
I've worked at some large companies, and none would accept that as a cost baseline for basic metrics. But if you said, "We'll cap it at $200k for 5,000 servers" then it gets more interesting.
The exceptions are if they're in a NewRelic or Splunk-type $$$$ deal:
1) they might save money by migrating or
2) they might use the lower cost to re-negotiate their Cadillac deals.
If you've used Cloudwatch at a medium or large-scale metrics and alerting, please post a comment how that worked out. I find their features/graphs too minimal, but I've never been forced to make it work as the sole metrics/monitoring/alerting tool.
(New managers always parrot, "Let's just use Cloudwatch!" their first month on the job, but then reality sinks in and any existing monitoring solution is by far the least of their problems. My favorite question to ask is, "So how do we migrate these existing dashboards that everybody uses? Ever done that?" followed by icy silence.)
Re: Amazon Managed Service for Prometheus
#65The pricing just for the ingest seems way off. $0.002 for 10,000 metrics might not seem like much by even a simple node_exporter will grab 700 metrics every 15 seconds. Thats $24/month just to ingest the cpu/ram/diskspace data from each server. Plus storage and query costs. At work I have a single r4.xlarge instance handling 1.3 million metrics every 15 seconds. Storage is not clustered but cost is only $500/month. I…
Pricing makes sense if you consider how Amazon operates at this point. You put basically a MVP product out there with abnormal pricing. Your enterprise customers that are drowning in money can start using it and using that money you can grow your org by hiring more engineers. At this point you start working on adding new features and do cost optimization. Since your whole architecture was designed based on "we have t…
I don’t think any company is drowning in money. Everyone has a budget they are working against. At the end of the day, you can bite an engineer or pay aws more. It’s all a trade off.
Re: Amazon Managed Service for Prometheus
#66Yeah I dunno about this, and the grafana service. They’re not exactly complicated to run on their own. At this pricing you may as well be on Datadog.
Scaling prometheus across multiple separate Kubernetes clusters is a fking nightmare.
Use Victoria metrics. One lightweight agent per cluster pushing to a centralised metrics store makes it so much easier.
Re: Amazon Managed Service for Prometheus
#67I very much dislike Prometheus, but the fact that AWS is offering it as a managed service means I am in the minority. I attribute much of Prometheus' success to the influence of ex-Googlers. They joined other companies, had a lot of clout, and sought out a tool that was similar to what they once used. I understand that the Google version of Prometheus is deprecated but there is no commercial equivalent.
What is in your opinion a better open source alternative to prometheus? Borgmon was inspiration for prometheus but was a totally different project so it is a complete rewrite
I feel like a broken record, but we are having great success with Victoria metrics as a drop in replacement.
Re: Amazon Managed Service for Prometheus
#68Earlier quoted context omitted.
Victoria Metrics is an absolutely superb drop in replacement.
It's not a drop-in replacement (even though it tries to sell itself as such), it's incompatible in a significant number of ways and throws away part of your data.
VictoriaMetrics (VM) is incompatible with Prometheus only in things which seemed to be "wrong" or "not good enough" in original. There was a very very long discussion about rate/irate implementation in Prometheus long before VM existence, and VM just fixed that and a bunch of other things.
And about "throws away part of your data" - let's be clear here. It is a precision loss while converting floating point numbers, not data dropping. Instead, user gets a superior compression and query performance which is a good trade off in my opinion.
For those who are interested in details of comparison VM vs Prometheus pls read the following article https://valyala.medium.com/evaluating-performance-and-correc...
Re: Amazon Managed Service for Prometheus
#69The pricing just for the ingest seems way off. $0.002 for 10,000 metrics might not seem like much by even a simple node_exporter will grab 700 metrics every 15 seconds. Thats $24/month just to ingest the cpu/ram/diskspace data from each server. Plus storage and query costs. At work I have a single r4.xlarge instance handling 1.3 million metrics every 15 seconds. Storage is not clustered but cost is only $500/month. I…
Pricing makes sense for enterprises. Considering that you may need a team (or a part of one) to maintain a self-hosted cluster at possibly 0.995 reliability, do upgrades, manage devs, run all the mandatory security scans, justify why some enterprise scan tool throwing errors isn't an issue, etc. Oh also justify why you need the manpower to do it, at which point your VP will tell you to just use the managed service.
It doesn’t though. I just did a cost projection on our estate and hiring two engineers to look after it on bare metal VMs is 30% cheaper than using the managed service. Plus it doesn’t require a lot of maintenance so we can use those guys on improving the product as well which actually gives direct customer benefits.
Re: Amazon Managed Service for Prometheus
#70Earlier quoted context omitted.
Victoria Metrics is an absolutely superb drop in replacement.
It's not a drop-in replacement (even though it tries to sell itself as such), it's incompatible in a significant number of ways and throws away part of your data.
VictoriaMetrics provides its own query lagnuage - MetricsQL. It is backwards compatible with PromQL with some exceptions. These exceptions are aimed towards solving practical issues experienced by users when working with PromQL. All these "incompatibilities" are well documented [1].