Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

11–20 of 142 posts

Re: Interview with Eric Brewer

#11

Post by mixmastamyk »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

See the shipping container analogy on why containers are so powerful and less work in the long run.

Re: Interview with Eric Brewer

#12

Post by menage »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Primarily because dedicated servers are a lot less efficient. The more different things you can pack on a machine while still ensuring that the high-priority/low-latency jobs get prompt access to the resources that they've reserved, the higher overall utilization you can achieve (and hence bring costs down),

At least with the Borg containers (I'm not familiar with how Omega and Kubernetes do things), there wasn't any additional layer of abstraction - the fact that there were multiple jobs running on the same kernel wasn't hidden from those jobs (although they didn't have to be aware of it). The containers were purely used for in-kernel resource accounting and control.

Re: Interview with Eric Brewer

#13

Post by mixmastamyk »

MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

> but it seems that running several linuxes on a linux machine is a bit much

Perhaps because that's not what happens, the host runs one copy of linux, which namespaces the containers.

Re: Interview with Eric Brewer

#14

Post by philip1209 »

mark_l_watson wrote: post #7

Great Interview! I worked as a contractor at Google in 2013 and loved their infrastructure. It was amazing to fire off a Borg job that used hundreds to thousands of servers, and the web based tools for tracking the job, fantastic logging to drill into problems, etc. And, Borg was two generations ago! Even though I am very happy doing what I am now, sometimes I literally wake up in the morning thinking about Google's…

How hard was it to learn these tools as a contractor? I hear that Google is more aggressively making their tools open-source to help the portability of their engineers' skills. Specifically, friends have told me that onboarding at Google is difficult because every tool is a proprietary one built on top of more proprietary systems. In addition, people who leave Google have trouble interviewing because they rely heavily on tools that are unavailable outside of Google. By beginning to contribute more to open-source, I think that it has the potential to make interviewing at and joining Google a smoother experience.

Re: Interview with Eric Brewer

#15

Post by mark_l_watson »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

I think that the answer maybe two fold: 1) resource utilization: it is more difficult with physical servers and costs more to run and has a larger environmental footprint. 2) it saves developer time to not worry about resources, OS upgrades, etc.

I like you main point however: I would like to know, given vistualization has X% overhead, what is X?

Re: Interview with Eric Brewer

#16

Post by mirceal »

DannoHung wrote: post #8
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'm completely unfamiliar with unikernels, but I just skimmed a brief description. My first question is: Do they still incur the overhead of running against a Hypervisor like common virtual machines? Also, possibly a clairification: Unless I'm misunderstanding what you mean by "running several linuxes on a linux machine", I believe you may be mistaken about the way containers work. Only one Linux is really running. A…

unikernel basically means that you pack your application and only the lower level bits and pieces that you need to get it working.

you no longer have an OS in the traditional sense.

Re: Interview with Eric Brewer

#17

Post by jacquesm »

menage wrote: post #12
jacquesm wrote: post #9

Earlier quoted context omitted.

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Primarily because dedicated servers are a lot less efficient. The more different things you can pack on a machine while still ensuring that the high-priority/low-latency jobs get prompt access to the resources that they've reserved, the higher overall utilization you can achieve (and hence bring costs down), At least with the Borg containers (I'm not familiar with how Omega and Kubernetes do things), there wasn't any…

> Primarily because dedicated servers are a lot less efficient

Assuming you're operating at scale I don't see why that would be the case. And if you're not, what's the point?

> The more different things you can pack on a machine while still ensuring that the high-priority/low-latency jobs get prompt access to the resources that they've reserved, the higher overall utilization you can achieve (and hence bring costs down),

Yes, that's the theory. But in practice you're assuming better static control over the situation than the operating system running multiple jobs will have over the dynamic situation. So you'll need to over-provision and then you're back to square one with your utilization or alternatively you'll under-provision and then you will run into performance issues. TANSTAAFL.

(For instance, what's to stop each container to ship another implementation of the same library as a dependency, say SSL).

Re: Interview with Eric Brewer

#18

Post by jacquesm »

DannoHung wrote: post #10
jacquesm wrote: post #9

Earlier quoted context omitted.

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Sorry, but what costs do containers incur? From my understanding, the resource overhead should be exceedingly minimal (disk space would ostensibly be the largest drawback, if you don't spend time cutting out the fat. Personally, I see this as a tooling issue since fat containers are completely orthogonal to how a container really executes). I understand some of the situation with IO isn't perfect yet, but I haven't h…

An OS has some pretty extensive insight into the processes that it executes, a container is an OS with a single application, so containers (assuming they take basic precautions for isolation) are not going to be able to schedule with anywhere near the efficiency that multiple processes on a single OS will.

I can see some (mostly potential at this point) security advantages but that's about it (and maybe those advantages will be enough to justify the performance overhead but containers are mostly treated as a silver bullet by the adherents and I'd like to see a bit more balance).

Re: Interview with Eric Brewer

#19

Post by jacquesm »

mixmastamyk wrote: post #11
jacquesm wrote: post #9

Earlier quoted context omitted.

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

See the shipping container analogy on why containers are so powerful and less work in the long run.

Software is very much unlike anything physical.

Re: Interview with Eric Brewer

#20

Post by DannoHung »

jacquesm wrote: post #18
DannoHung wrote: post #10

Earlier quoted context omitted.

Sorry, but what costs do containers incur? From my understanding, the resource overhead should be exceedingly minimal (disk space would ostensibly be the largest drawback, if you don't spend time cutting out the fat. Personally, I see this as a tooling issue since fat containers are completely orthogonal to how a container really executes). I understand some of the situation with IO isn't perfect yet, but I haven't h…

An OS has some pretty extensive insight into the processes that it executes, a container is an OS with a single application, so containers (assuming they take basic precautions for isolation) are not going to be able to schedule with anywhere near the efficiency that multiple processes on a single OS will. I can see some (mostly potential at this point) security advantages but that's about it (and maybe those advanta…

Do you know where I can read more about this? This is the first I have heard that processes running inside containers interfere with the kernel's scheduler. Or, sorry again, but maybe I'm not following your meaning in some way?
Post reply on HN