Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
1–10 of 49 posts
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#2I wonder if Google will ever open source it.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#3I wonder if Google will ever open source it.
Googler, opinions are my own.
Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have.
It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed by a former Google SRE that has since returned).
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#4I wonder if Google will ever open source it.
Trust me, it's not that impressive if you want to use it in your dozens of machines cluster...
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#5I wonder if Google will ever open source it.
Googler, opinions are my own. Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have. It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed b…
> Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released
Monarch might be an exception to that, though, because - as noted in the paper -, it has to be built without many dependencies, since almost everything else depends on it for monitoring and alerting.
That being said, I'm sure it would be extremely difficult to open source now since it was never built with open sourcing in mind.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#6Earlier quoted context omitted.
Googler, opinions are my own. Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have. It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed b…
> Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released Monarch might be an exception to that, though, because - as noted in the paper -, it has to be built without many dependencies, since almost everything else depends on it for monitoring and alerting. That being said, I'm sure it would be extremely difficult to…
It might not depend on many other services, but that doesn't preclude it depending on other code.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#7I wonder if Google will ever open source it.
Googler, opinions are my own. Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have. It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed b…
But when you publish a paper on Monarch, aren't you giving away the core idea?
In the case of PageRank, at least it was published after Google had already dominated the search space, i.e., many years after the conception, implementation, and utilization.
I wonder if Google papers are internally reviewed before publishing so as to make sure only partial information is revealed, and not the secret sauce.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#8Earlier quoted context omitted.
Googler, opinions are my own. Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have. It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed b…
But when you publish a paper on Monarch, aren't you giving away the core idea? In the case of PageRank, at least it was published after Google had already dominated the search space, i.e., many years after the conception, implementation, and utilization. I wonder if Google papers are internally reviewed before publishing so as to make sure only partial information is revealed, and not the secret sauce.
You could definitely build Monarch from this paper. It is very detailed. But keep in mind that Monarch is already ten years old.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#9Earlier quoted context omitted.
Googler, opinions are my own. Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have. It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed b…
But when you publish a paper on Monarch, aren't you giving away the core idea? In the case of PageRank, at least it was published after Google had already dominated the search space, i.e., many years after the conception, implementation, and utilization. I wonder if Google papers are internally reviewed before publishing so as to make sure only partial information is revealed, and not the secret sauce.
> But when you publish a paper on Monarch, aren't you giving away the core idea?
The thing is Monarch is not the secret sauce that makes Google money. Every other company in the world could be running monarch, and Google revenue and market position would not be affected.
Publicizing this is a win-win situation. The idea gets exposure and potentially other devs can be benefit, and Google gets some good PR as a place where devs get to work on cool stuff.
Re: Monarch: Google’s Planet-Scale In-Memory Time Series Database [pdf]
#10Interesting that this paper contains hard numbers hinting at Google's absolute scale. They say Monarch has 144000 leaves. Even if each leaf is assigned only 1 CPU core -- which is probably a low estimate because who would do that? -- that makes Google's monitoring stack a Top 100 supercomputer.
The only other places I've seen Google give out hard numbers were a presentation by Jeff Dean mentioning map-reduce core-years consumed per day, and a footnote in a paper that mentions how much CPU time Google Exacycle gives away to scientific computing every day. All of these calibration points are eye-opening.