Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grafana Mimir – Horizontally scalable long-term storage for Prometheus

grafana.com

Post reply on HN
Print view View original

31–40 of 121 posts

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#31

Post by bboreham »

k8sToGo wrote: post #24
bboreham wrote: post #17

Earlier quoted context omitted.

It’s a centralised multi-tenant store, supporting the Prometheus query API. So you can point clients directly at Mimir, they send in PromQL and they get data back in Json. (Note I work on Mimir)

But who does the scraping of the prometheus agents? Mimir or still prometheus server?

If you have systems exporting metrics in Prometheus style, then you can use Prometheus to scrape them and remote-write to Mimir.

You can alternately use Prometheus Agent, to save storing the data and running a query engine at the leaf.

You can also use the OpenTelemetry suite to perform the same operation, though this is more appealing if you want some other OpenTelemetry features at the same time. Eg if you prefer the ‘pipeline’ style.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#32

Post by outsb »

Thaxll wrote: post #27

So many solutions to the same problem, how does it compare to Victoria Metrics?

Given Victoria Metrics is the only solution I've seen to make data comparing it to other systems easily accessible as part of official documentation, it's the only one I pay attention to.

I knew from reading the docs what VM excelled at and areas it was weak in, long before I ever ran it (and expectations from running it matched the documentation). I hate aspirational marketing-saturated campaigns for deep tech projects where standards should obviously be higher, it speaks more about intended audience than it does the solution, and that's why in this respect VM is automatically a cut above the rest.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#33

Post by inkel »

k8sToGo wrote: post #24
bboreham wrote: post #17

Earlier quoted context omitted.

It’s a centralised multi-tenant store, supporting the Prometheus query API. So you can point clients directly at Mimir, they send in PromQL and they get data back in Json. (Note I work on Mimir)

But who does the scraping of the prometheus agents? Mimir or still prometheus server?

You configure with Remote Write [1] to the Mimir instance. Then the Prometheus agents will send the metrics to Mimir.

1: https://prometheus.io/docs/prometheus/latest/configuration/c...

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#34

Post by INTPenis »

cfors wrote: post #19

More engineering effort going into reinventing things that already exist to upsell people on Grafana cloud. What about focusing on the core value that Grafana provides, dashboards? Grafana 8 alerting is still in my opinion at a beta level. Dashboards as code has made no meaningful progress outside of community attempts in the past 3 years. The documentation for Grafana 8 alerts is still subpar. All of these things as…

What issues have you seen with Grafana alerting?

I'm curious because in my view it works so well that we abandoned alertmanager for Grafana alerts only well before v8.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#35

Post by jrwr »

maccard wrote: post #29
candiddevmike wrote: post #11

Earlier quoted context omitted.

Recently switched from their cloud service back to on-premise. The cloud version wasn't being updated and the entire setup experience left a lot to be desired with how you connect their on-premise grafana agent, especially if you aren't using their easy button deployment stuff. Also, billing for metrics is insane, as on any given day my metric load may vary between 5-7k or more. This caused some operational overhead…

> For $50/mo, you can self host everything easier, cheaper and with more control IMO. Can you give an example as to how you could self host a grafana stack for $50/month? On AWS that buys you 4 cores, 8GB memory and 0 storage, and it's certainly not easier than clicking one button on the grafana website.

Two low end Hetzner/OVH Boxes for redundancy should do the trick

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#36

Post by alexjplant »

maccard wrote: post #29
candiddevmike wrote: post #11

Earlier quoted context omitted.

Recently switched from their cloud service back to on-premise. The cloud version wasn't being updated and the entire setup experience left a lot to be desired with how you connect their on-premise grafana agent, especially if you aren't using their easy button deployment stuff. Also, billing for metrics is insane, as on any given day my metric load may vary between 5-7k or more. This caused some operational overhead…

> For $50/mo, you can self host everything easier, cheaper and with more control IMO. Can you give an example as to how you could self host a grafana stack for $50/month? On AWS that buys you 4 cores, 8GB memory and 0 storage, and it's certainly not easier than clicking one button on the grafana website.

There are Helm charts available for all Grafana products so if you already run a Kubernetes cluster and have spare capacity you can just throw it up there. Loki supports shipping logs to GCS/S3 natively and Prometheus can use Cortex (also available as a Helm chart) to do the same. Once you throw Grafana behind SSO and implement a backup cronjob you're done until you reach scale and have to start deploying/scaling individual components separately.

I implemented most of the above using Terraform on a managed DigitalOcean cluster on a Saturday a few months back; it wasn't super-hard. Alternatively you could rent a few VPSes someplace and use k3s or similar to get an unmanaged cluster.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#37

Post by notacoward »

netingle wrote: post #21
halfmatthalfcat wrote: post #5

How does this stack up with https://github.com/thanos-io/thanos , which I've used to pretty good success. The only criticism I have of Thanos though was the amount of moving pieces to maintain.

(Tom here; I started the Cortex project on which Mimir is based and lead the team behind Mimir) Thanos is an awesome piece of software, and the Thanos team have done a great job building an vibrant community. I'm a big fan - so much so we used Thanos' storage in Cortex. Mimir builds on this and makes it even more scalable and performance (with a sharded compactor and query engine). Mimir is multitenant from day 1, wh…

Multi-tenancy is something that shouldn't be underestimated. A lot of people think it's just a checklist item until (a) they need it or (b) they try to implement it in an existing system. Kudos for making it a day-one feature.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#38

Post by SuperQue »

One interesting question I have is regards to global availability.

With our current Thanos deployment, we can tie a single geo regional deployment together with a tiered query engine.

Basically like this:

"Global Query Layer" -> "Zone Cluster Query Layer" -> "Prom Sidecar / Thanos Store"

We can duplicate the "Global Query Layer" in multiple geo regions with their own replicated Grafana instances. If a single region/zone has trouble we can still access metrics in other regions/zones. This avoids Thanos having any SPoFs for large multi-user(Dev/SRE) orgs.

Re: Grafana Mimir – Horizontally scalable long-term storage for Prometheus

#39

Post by berkes »

cfors wrote: post #19

More engineering effort going into reinventing things that already exist to upsell people on Grafana cloud. What about focusing on the core value that Grafana provides, dashboards? Grafana 8 alerting is still in my opinion at a beta level. Dashboards as code has made no meaningful progress outside of community attempts in the past 3 years. The documentation for Grafana 8 alerts is still subpar. All of these things as…

Seconded. While I like the idea of Grafana, and use it for some projects, it lacks features in the graphing and dashboarding part. I too presumed this is because they are spending more on backends, pipelines and collection..

I don't need more backends, pipelines or collections. I need a frontend to display the data that I have (in backends) already.

I need to:

* Be able to pipe KPIs into a storage. Doesn't need big-data, high-volume, or extreme granularity. OR

* Have grafana grab data from an API/HTTP endpoint. It does this with prometheus just fine.

* Have a way to insert some of my own figures. Currently I wire up some google-sheet to grafana and fill that. I always have some data that I cannot or will not (yet) grab automatically. Like "amount of hours spent working on project" or "MRR" or such.

Its possible with Grafana. But the experience is subpar, the tweaking and wiggling is big and the outcome is an OK-ish, but not too convincing dashboard. I'm convinced an alternative that tackles this better (for niches) will eat into grafana.

Post reply on HN