101–110 of 121 posts
Is this the project you guys referenced using Apache Arrow for?
I don't think so! I think thats being used in Tempo, but I'm not sure.
However, we are currently primarily targeting Parquet as our columnar format in object storage.
Expect an announcement soon!
One interesting question I have is regards to global availability. With our current Thanos deployment, we can tie a single geo regional deployment together with a tiered query engine. Basically like this: "Global Query Layer" -> "Zone Cluster Query Layer" -> "Prom Sidecar / Thanos Store" We can duplicate the "Global Query Layer" in multiple geo regions with their own replicated Grafana instances. If a single region/z…
It's really a shame that Loki in particular doesn't share this kind of architecture. Seems like Mimir, frustratingly, will share this deficiency.
You can be scalable, and still cost a lot of money to scale out. Unit economics are important.
Grafana Labs needs to make a convincing comparison chart of some kind between Mimir, Thanos, and Cortex. Thanos and Cortex are both mature projects and are both CNCF Incubating projects. Why would anyone switch to a new prometheus long-term storage solution from those? *EDIT* : I see from another reply there is a basic comparison to Cortex here: https://grafana.com/blog/2022/03/30/announcing-grafana-mimir... To the M…
You're forgetting VictoriaMetrics that's presumably the best choice for Prometheus long term storage. Such a solid solution exists and yet another competitor? Not sure why they didn't just buy VictoriaMetrics and possibly rebrand it.
But Grafana labs, employ lots of people that have worked on Cortex since its inception at weaveworks, and has developed strong in-house knowledge about it so Grafana is fully commited to Cortex (now Mimir) and have developed derivatives for logs (Loki) and traces (Tempo) heavily based on the Cortex model.
Earlier quoted context omitted.
(Tom here; I started the Cortex project on which Mimir is based and lead the team behind Mimir) Thanos is an awesome piece of software, and the Thanos team have done a great job building an vibrant community. I'm a big fan - so much so we used Thanos' storage in Cortex. Mimir builds on this and makes it even more scalable and performance (with a sharded compactor and query engine). Mimir is multitenant from day 1, wh…
Okay, but why? I am using Thanos today. It works, it's complex, when it breaks, it's a bit of a challenge to fix, but it happens. It doesn't break often. It does the job. Mimir, which is based on Cortex, using either Mimir, or Cortex, what benefit am I getting? I get asked every few months about moving off of Thanos to Cortex, and today now Mimir, and I don't have any substantial reason to do so. It feels like moving…
How does it work with Rules? So far I cannot see if this can be a replacement for prometheus since I cannot see how can we re-use our prometheus rules with Mimir. Anyone knows anything around that?
Mimir includes a ruler component, which is responsibile to evaluate Prometheus recording and alerting rules. It also exposes a set of APIs to configure the rule groups. For example, you can use this API to upload a rule group: https://grafana.com/docs/mimir/latest/operators-guide/refere... Mimir is released with a CLI tool called "mimirtool" which, among other things, allow you to configure the rule groups (under the…
I'm running a single Victoriametrics instance which has 230bn metrics, consuming ~4GB of memory and barely 200m of CPU utilization (only spikes to ~1.5cores when it flushes these datapoints from RAM to disk). I've previously[1] shared my experience of setting up Victoriametrics for long term Prometheus storage back in 2020 and since then this product has just kept getting better.
Over time, I switched to `vmagent` and `vmalert` as well which offer some nice little things (like did you know, you can't break up the scrape config of Prometheus into multiple files? `vmagent` does that happily). The whole setup is very easy to manage for an Ops person (as compared to Thanos/Cortex. Yet to checkout Mimir though!) as well. I've barely had to tweak any default configs that come in Victoriametrics and I even increased the retention of metrics from a month to multiple months after gaining confidence in prod.
Can we use Prometheus/Mimir as general purpose time series database? Prometheus is built for monitoring purposes and may not be for general purpose time series databases like InfluxDB (I am hoping to be wrong). What are the disadvantages/limitations for using Prometheus/Mimir as general purpose time series database?
Grafana Labs needs to make a convincing comparison chart of some kind between Mimir, Thanos, and Cortex. Thanos and Cortex are both mature projects and are both CNCF Incubating projects. Why would anyone switch to a new prometheus long-term storage solution from those? *EDIT* : I see from another reply there is a basic comparison to Cortex here: https://grafana.com/blog/2022/03/30/announcing-grafana-mimir... To the M…