Live data from Hacker News

Grafana Mimir – Horizontally scalable long-term storage for Prometheus

grafana.com

71–80 of 121 posts

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#71

Earlier quoted context omitted.

What issues have you seen with Grafana alerting? I'm curious because in my view it works so well that we abandoned alertmanager for Grafana alerts only well before v8.

How did you define alarms as code in a practical way before v8? and after?

Did not tbh. We have an ops department that do not complain about menial tasks.

But of course IaC is the way we must follow.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#72

Earlier quoted context omitted.

What issues have you seen with Grafana alerting? I'm curious because in my view it works so well that we abandoned alertmanager for Grafana alerts only well before v8.

How did you define alarms as code in a practical way before v8? and after?

Hi, I work on Grafana Alerting. Provisioning of alert rules (and other objects used for alerting) will be possible using a new API in Grafana 8.5 and we will update the Grafana Terraform provider right after to take advantage of this new API.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#73
post #44

Earlier quoted context omitted.

I'm not seeing a comparison to Thanos

Why would you? Parent says its a comparison of Mimir and Cortex.

Re-read the full thread...

>>Grafana Labs needs to make a convincing comparison chart of some kind between Mimir, Thanos, and Cortex.

>I agree! Which is why I put one in the blog post ;-)

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#75

Earlier quoted context omitted.

(Tom here; I started the Cortex project on which Mimir is based and lead the team behind Mimir) Thanos is an awesome piece of software, and the Thanos team have done a great job building an vibrant community. I'm a big fan - so much so we used Thanos' storage in Cortex. Mimir builds on this and makes it even more scalable and performance (with a sharded compactor and query engine). Mimir is multitenant from day 1, wh…

Multi-tenancy is something that shouldn't be underestimated. A lot of people think it's just a checklist item until (a) they need it or (b) they try to implement it in an existing system. Kudos for making it a day-one feature.

While I agree with your point in the general case, would you mind elaborating on the specific case of Prometheus?

My understanding is that the recommended best-practice for Prometheus is to deploy as many of them as necessary, as close to the monitored infrastructure as possible.

What use case would require deploying a single Mimir, so supposedly Prometheus (cluster) in the case of serving multiple tenants? Why not just deploy a dedicated Prometheus / Mimir stack per client?

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#76

One interesting question I have is regards to global availability. With our current Thanos deployment, we can tie a single geo regional deployment together with a tiered query engine. Basically like this: "Global Query Layer" -> "Zone Cluster Query Layer" -> "Prom Sidecar / Thanos Store" We can duplicate the "Global Query Layer" in multiple geo regions with their own replicated Grafana instances. If a single region/z…

The typical way to run Mimir is centralised, with different regions/datacenters feeding metrics in to one place. You can run that central system across multiple AZs.

If you run Mimir with an object store (e.g. S3) that supports replication then you can have copies in multiple geographies and query them, but the copies will not have the most recent data.

(Note I work on Mimir)

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#77

Is this the project you guys referenced using Apache Arrow for?

Maybe you're thinking of this - the data structure used by datasources for Grafana dashboards:

https://grafana.com/docs/grafana/latest/developers/plugins/d...

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#78
post #32
post #27

So many solutions to the same problem, how does it compare to Victoria Metrics?

Given Victoria Metrics is the only solution I've seen to make data comparing it to other systems easily accessible as part of official documentation, it's the only one I pay attention to. I knew from reading the docs what VM excelled at and areas it was weak in, long before I ever ran it (and expectations from running it matched the documentation). I hate aspirational marketing-saturated campaigns for deep tech proje…

Cortex, Thanos and Mimir all support "remote-read" protocol (documented in Prometheus: https://prometheus.io/docs/prometheus/latest/storage/#remote...), so external systems (eg Prometheus) can read data from them easily.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#79
post #19

More engineering effort going into reinventing things that already exist to upsell people on Grafana cloud. What about focusing on the core value that Grafana provides, dashboards? Grafana 8 alerting is still in my opinion at a beta level. Dashboards as code has made no meaningful progress outside of community attempts in the past 3 years. The documentation for Grafana 8 alerts is still subpar. All of these things as…

What issues have you seen with Grafana alerting? I'm curious because in my view it works so well that we abandoned alertmanager for Grafana alerts only well before v8.

Grafana alerts (before version 8) worked great. We use them, but the Grafana 8 alerting features are half-baked at best.

* Grafana 8 alerts removed the Image Preview, which was extremely useful during issues. [0]

* Grafana 8 alerts don't have any way of being stored as code. In fact the API that they provide in their docs [0][1] doesn't work, or isn't up to date.

* The expression languages have zero documentation about them, so aren't exactly useful for things that might get a developer out of bed in the middle of the night.

[0] https://github.com/grafana/grafana/discussions/38030#discuss...

[1] https://editor.swagger.io/?url=https://raw.githubusercontent...

[2] https://community.grafana.com/t/posting-an-alert-using-grafa...

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#80

Earlier quoted context omitted.

How did you define alarms as code in a practical way before v8? and after?

Did not tbh. We have an ops department that do not complain about menial tasks. But of course IaC is the way we must follow.

Building a dashboard by clickety/clacking around is not a menial task, consistency across dashboards is a a core unit of observability to ensure x-functional teams can discuss issues across a common language/viewpoint, which is only enforceable through a declarative dashboard syntax.
Post reply on HN