Live data from Hacker News

Monarch: Google’s Planet-Scale In-Memory Time Series Database

micahlerner.com

51–60 of 133 posts

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#51

Interesting that Google replaced a pull based metric system similar to Prometheus with a push based system... I thought one of the selling points of Prometheus and the pull based dance was how scalable it was?

Pull collection eventually became a real scaling bottleneck for Monarch.

The way the "pull" collection worked was that there was an external process-discovery mechanism, which the leaf used to connect to the entities it was monitoring, the leaf backend processes would connect to the monitored entities to an endpoint that the collection library would listen on, and those entities collection libraries would stream the metric measurements according to the schedules that the leaves sent.

Several problems.

First, the leaf-side data structures and TCP connections become very expensive. If that leaf process is connecting to many many many thousands of monitored entities, TCP buffers aren't free, keep-alives aren't free, and a host of other data structures. Eventually this became an...interesting...fraction of the CPU and RAM on these leaf processes.

Second, this implies a service discovery mechanism so that the leaves can find the entities to monitor. This was a combination of code in Monarch and an external discovery service. This was a constant source of headaches an outages, as the appearance and disappearance of entities is really spiky and unpredictable. Any burp in operation of the discovery service could cause a monitoring outage as well. Relatedly, the technical "powers that be" decided that the particular discovery service, of which Monarch was the largest user, wasn't really something that was suitable for the infrastructure at scale. This decision was made largely independently of Monarch, but required Monarch to move off.

Third, Monarch does replication, up to three ways. In the pull-based system, it wasn't possible to guarantee that the measurement that each replica sees is the same measurement with the same microsecond timestamp. This was a huge data quality issue that made the distributed queries much harder to make correct and performant. Also, the clients had to pay both in persistent TCP connections on their side and in RAM, state machines, etc., for this replication as a connection would be made from each backend leaf processes holding a replica for a given client.

Fourth, persistent TCP connections and load balancers don't really play well together.

Fifth, not everyone wants to accept incoming connections in their binary.

Sixth, if the leaf process doesn't need to know the collection policies for all the clients, those policies don't have to be distributed and updated to all of them. At scale this matters for both machine resources and reliability. This can be made a separate service, pushed to the "edge", etc.

Switching from a persistent connection to the clients pushing measurements in distinct RPCs as they were recorded eventually solved all of these problems. It was a very intricate transition that took a long time. A lot of people worked very hard on this, and should be very proud of their work. I hope some of them jump in to the discussion! (At very least they'll add things I missed/didn't remember... ;^)

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#52
post #46

Earlier quoted context omitted.

It is not. It basically does the opposite of what every scalable system does. To get HA you double you’re number of pollers. To get scale your queries you aggregate them into other prometheii. If this is scalability: everything is scalable.

I don't understand how the properties you're describing imply that Prometheus isn't scalable. High Availability always requires duplication of effort. Scaling queries always requires sharding and aggregation at some level. I've deployed stock Prometheus at global scale, O(100k) targets, with great success. You have to understand and buy into Prometheus' architectural model, of course.

The ways in which you can scale Prometheus: you can scale anything.

It does not; itself, have highly scalable properties built in.

It does not do sharding, it does not do proxying, it does not do batching, it does not do anything that would allow it to run multiple servers and query over multiple servers.

Look. I’m not saying that it doesn’t work; but when I read about borgmon and Prometheus: I understood the design goal was intentionally not to solve these hard problems, and instead use them as primitive time series systems that can be deployed with a small footprint basically everywhere (and individually queried).

I submit to you, I could also have an influxdb in every server and get the same “scalability”.

Difference being that I can actually run a huge influxdb cluster with a dataset that exceeds the capabilities of a single machine.

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#53
post #46

Earlier quoted context omitted.

It is not. It basically does the opposite of what every scalable system does. To get HA you double you’re number of pollers. To get scale your queries you aggregate them into other prometheii. If this is scalability: everything is scalable.

I don't understand how the properties you're describing imply that Prometheus isn't scalable. High Availability always requires duplication of effort. Scaling queries always requires sharding and aggregation at some level. I've deployed stock Prometheus at global scale, O(100k) targets, with great success. You have to understand and buy into Prometheus' architectural model, of course.

Lots of systems provide redundancy with 2X cost. It's not that hard.

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#54
post #52

Earlier quoted context omitted.

I don't understand how the properties you're describing imply that Prometheus isn't scalable. High Availability always requires duplication of effort. Scaling queries always requires sharding and aggregation at some level. I've deployed stock Prometheus at global scale, O(100k) targets, with great success. You have to understand and buy into Prometheus' architectural model, of course.

The ways in which you can scale Prometheus: you can scale anything. It does not; itself, have highly scalable properties built in. It does not do sharding, it does not do proxying, it does not do batching, it does not do anything that would allow it to run multiple servers and query over multiple servers. Look. I’m not saying that it doesn’t work; but when I read about borgmon and Prometheus: I understood the design…

It seems like you're asserting a very specific definition of scalability that excludes Prometheus' scalability model. Scalability is an abstract property of a system that can be achieved in many different ways. It doesn't require any specific model of sharding, batching, query replication, etc. Do you not agree?

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#55

Earlier quoted context omitted.

In my experience, Monarch storing histograms and being unable to rebucket on the fly is a big problem. A percentile line on a histogram will be incredibly misleading, because it's trying to figure out what the p50 of a bunch of buckets is. You'll see monitoring artifacts like large jumps and artificial plateaus as a result of how requests fall into buckets. The bucketer on the default RPC latency metric might not be…

I definitely remember a lot of time spent tweaking histogram buckets for performance vs. accuracy. The default bucketing algorithm at the time was powers of 4 or something very unusual like that.

It's because powers of four was great for the original application of statistics on high traffic services where the primary thing the user was interested in was deviations from the norm, and with a high traffic system the signal for what the norm is would be very strong.

I tried applying it to a service with much lower traffic and found the bucketing to be extremely fussy.

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#56
post #52

Earlier quoted context omitted.

The ways in which you can scale Prometheus: you can scale anything. It does not; itself, have highly scalable properties built in. It does not do sharding, it does not do proxying, it does not do batching, it does not do anything that would allow it to run multiple servers and query over multiple servers. Look. I’m not saying that it doesn’t work; but when I read about borgmon and Prometheus: I understood the design…

It seems like you're asserting a very specific definition of scalability that excludes Prometheus' scalability model. Scalability is an abstract property of a system that can be achieved in many different ways. It doesn't require any specific model of sharding, batching, query replication, etc. Do you not agree?

I’m not defining terms arbitrarily.

https://en.wikipedia.org/wiki/Database_scalability

Scalability means running a single workload across multiple machines.

Prometheus intentionally does not scale this way.

I’m not being mean, it is fact.

It has made engineering design trade offs and one of those means it is not built to scale, this is fine, I’m not here pooping on your baby.

You can build scalable systems on top of things which do not individually scale.

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#57
post #52

Earlier quoted context omitted.

The ways in which you can scale Prometheus: you can scale anything. It does not; itself, have highly scalable properties built in. It does not do sharding, it does not do proxying, it does not do batching, it does not do anything that would allow it to run multiple servers and query over multiple servers. Look. I’m not saying that it doesn’t work; but when I read about borgmon and Prometheus: I understood the design…

It seems like you're asserting a very specific definition of scalability that excludes Prometheus' scalability model. Scalability is an abstract property of a system that can be achieved in many different ways. It doesn't require any specific model of sharding, batching, query replication, etc. Do you not agree?

Prometheus cannot evaluate a query over time series that do not fit in the memory of a single node, therefore it is not scalable.

The fact that it could theoretically ingest an infinite amount of data that it cannot thereafter query is not very interesting.

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#58
post #50

Earlier quoted context omitted.

Thats just like.. you know.. their opinion, man. https://prometheus.io/docs/prometheus/latest/federation/

Prometheus federation isn't distributed evaluation. It "federates" from other nodes onto a single node. > Federation allows a Prometheus server to scrape selected time series from another Prometheus server

Collect directly from shard-level prometheus then aggregate using /federate at another level. That’s how thanos also works afaik

Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database

#60

A huge difference between monarch and other tsdb that isn’t outlined in this overview, is that a storage primitive for schema values is a histogram. Most (maybe all besides Circonus) tsdb try to create histograms at query time using counter primitives. All of those query time histogram aggregations are making pretty subtle trade offs that make analysis fraught.

I've been pretty happy with datadog's distribution type [1] that uses their own approximate histogram data structure [2]. I haven't evaluated their error bounds deeply in production yet, but I haven't had to tune any bucketing. The linked paper [3] claims a fixed percentage of relative error per percentile.

[1] https://docs.datadoghq.com/metrics/distributions/

[2] https://www.datadoghq.com/blog/engineering/computing-accurat...

[3] https://arxiv.org/pdf/1908.10693.pdf

Post reply on HN