Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large-scale cluster management at Google with Borg

research.google.com

Post reply on HN
Print view View original

31–37 of 37 posts

Re: Large-scale cluster management at Google with Borg

#31

Post by kozyraki »

All of us who have used Borg over the years are very appreciative of the technology and its capabilities. Congrats to all the Googlers, current and ex, that contributed to it. This paper should be toward the top of the reading list for anyone working on the topic.

Nevertheless, there are many open questions for large-scale cluster management for researchers and developer to address. Here are some of my favorite: - The curse of overprovisioning: Borg and many other systems rely on reservation which are systematically exaggerated by users. Right sizing these reservations is one way to go beyond the 40-50% usage shown in the Borg paper (see fig 12). A promising way of doing this is Christina Delimitrou's work using classification techniques (see http://goo.gl/vFf8oN) - Oversubscription using better isolation mechanisms): this is what the Borg paper calls resource reclamation. Take unused (but reserved) resources from priority jobs and use them for best effort analytics. David Lo (http://web.stanford.edu/~davidlo/) has a very interesting paper coming up on how to coordinate cpu sets, cache partitioning, Linux TC, RAPL/DVFS (power management) to run websearch clusters at >90% by packing them with analytics without causing ANY glitch on search. And that is Google search.

There are definitely more interesting. Exciting times.

Re: Large-scale cluster management at Google with Borg

#33

Post by mdaniel »

kozyraki wrote: post #31

All of us who have used Borg over the years are very appreciative of the technology and its capabilities. Congrats to all the Googlers, current and ex, that contributed to it. This paper should be toward the top of the reading list for anyone working on the topic. Nevertheless, there are many open questions for large-scale cluster management for researchers and developer to address. Here are some of my favorite: - Th…

> http://goo.gl/vFf8oN

goes to: http://web.stanford.edu/~cdel/2014.asplos.quasar.pdf

It's my understanding that URL shorteners are frowned upon in HN posts or comments.

Re: Large-scale cluster management at Google with Borg

#34

Post by mark_l_watson »

Nice writeup - I just saved the PDF in my keep for ever PDF repo.

When I contracted at Google at 2013 I loved their infrastructure. For my task I had to run huge Borg jobs and the job submission, monitoring and logging system were very easy to use. I really liked the summary of hardware failures that occurred - hardware really is not very reliable when running at scale.

After not using AppEngine for a few years I have started using it recently for two personal projects. Using AppEngine's logging and scaling features is a tiny bit like using Google's internal infrastructure - makes me a little nostalgic.

Re: Large-scale cluster management at Google with Borg

#35

Post by scott_s »

juliangregorian wrote: post #27
dekhn wrote: post #25

Earlier quoted context omitted.

It was a secret. You weren't allowed to mention borg outside Google. Note John Wilkes was generally careful about not saying the name was 'Borg' or acknowledging it as such and instead referred to 'Omega'.

Perhaps you were told it was secret. It was not secret. Here's one of the articles I was referring to. http://www.wired.com/2013/03/google-borg-twitter-mesos/ John Wilkes mentions Borg, acts coy about the name for some reason (seems like a pattern), and mentions Omega, its nascent replacement.

Perhaps you two are using different definitions of the word "secret". You are probably using the word to mean "a fact that no one outside of a select group of people knows". dekhn is probably using it to mean "a fact which my company has said we are not allowed to publicly acknowledge".

Re: Large-scale cluster management at Google with Borg

#36

Post by kozyraki »

mdaniel wrote: post #33
kozyraki wrote: post #31

All of us who have used Borg over the years are very appreciative of the technology and its capabilities. Congrats to all the Googlers, current and ex, that contributed to it. This paper should be toward the top of the reading list for anyone working on the topic. Nevertheless, there are many open questions for large-scale cluster management for researchers and developer to address. Here are some of my favorite: - Th…

> http://goo.gl/vFf8oN goes to: http://web.stanford.edu/~cdel/2014.asplos.quasar.pdf It's my understanding that URL shorteners are frowned upon in HN posts or comments.

I learn something every day :)
Post reply on HN