CNCF's Cortex v1.0: scalable, fast Prometheus implementation • Hacker News Board

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#21

Post by cat199 » Thu, Apr 02, 2020, 2:59 PM UTC

netingle wrote: post #12

ones_and_zeros wrote: post #10

Isn't prometheus an implementation and not an interface? I have "prometheus" running in my cluster, if it's not cortex, what implementation am I using?

Yes, Prometheus is an implementation - the HN text has a limited number of words, so I thought "Prometheus implementation" conveyed the fact Cortex was trying to be a 100% API compatible implementation of Prometheus, but with scalability, replication etc

how about:

CNCF's Cortex v1.0: scalable, fast Prometheus API implementation ready for prod (grafana.com)

saves 1 char.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#22

Post by Rapzid » Thu, Apr 02, 2020, 4:46 PM UTC

Dat architecture tho: https://cortexmetrics.io/docs/architecture/ . Holy bi-gebus.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#23

Post by netingle » Thu, Apr 02, 2020, 4:48 PM UTC

Rapzid wrote: post #22

Dat architecture tho: https://cortexmetrics.io/docs/architecture/ . Holy bi-gebus.

Thats the "microservices" mode - you can run it as a single process and the architecture becomes super boring.

Its like looking at the module interdependencies of reasonably large piece of software; of course its going to look complicated.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#24

Post by outworlder » Thu, Apr 02, 2020, 5:45 PM UTC

ones_and_zeros wrote: post #10

Isn't prometheus an implementation and not an interface? I have "prometheus" running in my cluster, if it's not cortex, what implementation am I using?

You are using Prometheus.

However, Prometheus can use different storage backends. The TSDB that it comes with is horrible.

I mean, it's workable. And can store an impressive amount of data points. If you don't care about historical data or scale, it may be all you need.

However, if your scale is really large, or if you care about the data, it may not be the right solution, and you'll need something like Cortex.

For instance, Prometheus' own TSSB has no 'fsck'-like tool. From time to time, it does compaction operations. If your process (or pod in K8s) dies, you may be left with duplicate time series. And now you have to delete some (or a lot!) of your data to recover.

Prometheus documentation, last I checked, even says it is not suitable for long-term storage.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#25

Post by nopzor » Thu, Apr 02, 2020, 5:46 PM UTC

awesome job by the cortex team!

there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs.

- at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fact, cortex vendors in prometheus.

- you can think of cortex as a scale-out, multi-tenant, highly available "implementation" of prometheus itself.

- the reason grafana labs put so much resources into cortex is because it powers our grafana cloud product (which offers a prometheus backend). like grafana itself, we are also actively working on an enterprise edition of cortex that is designed to meet the security and feature requirements of the largest companies in the world.

- yes, cortex was born at weaveworks in 2016. tom wilkie (vp of product at grafana labs) co-created it while he worked there. after tom joined grafana labs in 2018, we decided to pour a lot more resources into the project, and managed to convince weave.works to move it to the cncf. this was a great move for the project and the community, and cortex has come a long long way in the last 2 years.

once again, a big hat tip to everyone who made this release possible. a big day for the project, and for prometheus users in general!

[edit: typos]

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#26

Post by Florin_Andrei » Thu, Apr 02, 2020, 6:05 PM UTC

nopzor wrote: post #25

awesome job by the cortex team! there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs. - at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fac…

I'm worried about this statement:

> Local storage is explicitly not production ready at this time.

https://cortexmetrics.io/docs/getting-started/getting-starte...

But I want a scale-out, multitenant implementation of Prometheus with local storage that's ready for prod. What are my options then? VictoriaMetrics?

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#27

Post by m0rphling » Thu, Apr 02, 2020, 6:12 PM UTC

nopzor wrote: post #25

awesome job by the cortex team! there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs. - at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fac…

Please note the difference between complimentary and complementary. It's a common homophone confusion in English.

The former means free or charge or expressing praise or a compliment.

The latter means disparate things go well together and enhance each others' qualities.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#28

Post by nopzor » Thu, Apr 02, 2020, 6:27 PM UTC

m0rphling wrote: post #27

nopzor wrote: post #25

awesome job by the cortex team! there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs. - at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fac…

Please note the difference between complimentary and complementary . It's a common homophone confusion in English. The former means free or charge or expressing praise or a compliment. The latter means disparate things go well together and enhance each others' qualities.

thanks for the complimentary tip ;) fixed.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#29

Post by netingle » Thu, Apr 02, 2020, 6:28 PM UTC

Florin_Andrei wrote: post #26

nopzor wrote: post #25

awesome job by the cortex team! there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs. - at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fac…

I'm worried about this statement: > Local storage is explicitly not production ready at this time. https://cortexmetrics.io/docs/getting-started/getting-starte... But I want a scale-out, multitenant implementation of Prometheus with local storage that's ready for prod. What are my options then? VictoriaMetrics?

There are a bunch of different solutions out there; Thanos, Influx, federated Prometheus etc.

The local Cortex storage works pretty well but we have a very high bar for production worthiness. Right now I'd recommend using Bigtable of DynamoDB, and if you're on premise Cassandra. In the future the block storage will allow you to run minio.

Re: CNCF's Cortex v1.0: scalable, fast Prometheus implementation

#30

Post by gouthamve » Thu, Apr 02, 2020, 6:29 PM UTC

Florin_Andrei wrote: post #26

nopzor wrote: post #25

awesome job by the cortex team! there's a lot of good questions, and some confusion in this thread. here is my view. note: i'm definitely biased; am the co-founder/ceo at grafana labs. - at grafana labs we are huge fans of prometheus. it has become the most popular metrics backend for grafana. we view cortex and prometheus as complementary. we are also very active contributors to the prometheus project itself. in fac…

I'm worried about this statement: > Local storage is explicitly not production ready at this time. https://cortexmetrics.io/docs/getting-started/getting-starte... But I want a scale-out, multitenant implementation of Prometheus with local storage that's ready for prod. What are my options then? VictoriaMetrics?

The only one I know with "non-experimental" local-storage is VictoriaMetrics. But the big thing there is that data in VM is not replicated, so when you lose a disk/node, you lose that data.

Having said that, both Thanos and Cortex have experimental local-storage modes that are pretty good. You could also try them for now while they get production ready.