Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large-scale cluster management at Google with Borg

research.google.com

Post reply on HN
Print view View original

1–10 of 37 posts

Re: Large-scale cluster management at Google with Borg

#2

Post by WestCoastJustin »

There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with.

[1] https://www.youtube.com/watch?v=0ZFMlO98Jkc

Edit: remove error in my comment re: borg/omega order.

Re: Large-scale cluster management at Google with Borg

#3

Post by nostrademons »

WestCoastJustin wrote: post #2

There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with. [1] https://www.youtube.com/watch?v=0ZFMlO98Jkc Edit: remove error in my comment re: borg/omega order.

Omega actually comes after Borg (hence the name). It's just that Borg was quite confidential until now. I'm pretty surprised they let the name slip.

Re: Large-scale cluster management at Google with Borg

#4

Post by WestCoastJustin »

nostrademons wrote: post #3
WestCoastJustin wrote: post #2

There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with. [1] https://www.youtube.com/watch?v=0ZFMlO98Jkc Edit: remove error in my comment re: borg/omega order.

Omega actually comes after Borg (hence the name). It's just that Borg was quite confidential until now. I'm pretty surprised they let the name slip.

Ah, thanks! Updated my comment.

Re: Large-scale cluster management at Google with Borg

#5

Post by tonfa »

WestCoastJustin wrote: post #2

There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with. [1] https://www.youtube.com/watch?v=0ZFMlO98Jkc Edit: remove error in my comment re: borg/omega order.

It is the other way: Omega was presented as the successor of Borg.

Re: Large-scale cluster management at Google with Borg

#6

Post by tonfa »

nostrademons wrote: post #3
WestCoastJustin wrote: post #2

There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with. [1] https://www.youtube.com/watch?v=0ZFMlO98Jkc Edit: remove error in my comment re: borg/omega order.

Omega actually comes after Borg (hence the name). It's just that Borg was quite confidential until now. I'm pretty surprised they let the name slip.

The name was public since Omega was publicized. The paper itself is great, there's everything about Borg in one place (scheduling, quota, isolation, etc).

Re: Large-scale cluster management at Google with Borg

#7

Post by obstinate »

Oh, fun, they finally published this. How many times I've had to catch myself from saying the word "Borg" I don't even know.

The performance isolation is good, but I wouldn't really seek after high utilization unless compute costs are significant to your business. We've seen some crazy things where nominally non-interfering jobs cause significant performance degradation to other jobs on the same node. There's work yet to do here.

Re: Large-scale cluster management at Google with Borg

#8

Post by bbrazil »

> cc would be reachable via 50.jfoo.ubar.cc.borg.google.com.

I've implemented similar at my current job, as that sort of naming is very convenient. http://www.boxever.com/using-google-apps-openid-connect-with... has a sketch of how to do this with Apache as a reverse proxy with Google Auth, though we're using a PAC file now going to a HTTPS forward proxy to avoid limitations of SSL wildcard certs.

Post reply on HN