Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Eric Brewer

medium.com

Post reply on HN
Print view View original

41–50 of 142 posts

Re: Interview with Eric Brewer

#41

Post by jacobsenscott »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

The "special snowflake" problem is the primary reason VMs and containers are better - http://martinfowler.com/bliki/SnowflakeServer.html

While VMs and containers in VMs don't utilize the hardware as efficiently as a dedicated server they make your people much more efficient and happy. Unless you are running at a fairly large scale making your people more efficient gives you a much better ROI than making your servers more efficient.

Re: Interview with Eric Brewer

#42

Post by bcantrill »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Containers have actually been around for a long, long time -- and have well-known operational efficiencies. So this isn't a "shiny new layer-of-abstraction", it's a tried-and-true abstraction that has been operating in production and at scale for a the better part of the last decade.[1] That said, the developer fascination with containers (which is to say, Docker) is new, and there is a bit of a wild west of abstraction around up-stack abstractions -- but that confusion shouldn't be conflated with the abstraction of OS-based virtualization, which remains a clear improvement over HW-based virtualization and the next logical step function in infrastructure deployment.

[1] https://www.joyent.com/developers/videos/docker-and-the-futu...

Re: Interview with Eric Brewer

#43

Post by jacquesm »

jacobsenscott wrote: post #41
jacquesm wrote: post #9

Earlier quoted context omitted.

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

The "special snowflake" problem is the primary reason VMs and containers are better - http://martinfowler.com/bliki/SnowflakeServer.html While VMs and containers in VMs don't utilize the hardware as efficiently as a dedicated server they make your people much more efficient and happy. Unless you are running at a fairly large scale making your people more efficient gives you a much better ROI than making your servers…

That's the first explanation that actually makes sense to me, thank you.

Re: Interview with Eric Brewer

#44

Post by davmre »

mark_l_watson wrote: post #7

Great Interview! I worked as a contractor at Google in 2013 and loved their infrastructure. It was amazing to fire off a Borg job that used hundreds to thousands of servers, and the web based tools for tracking the job, fantastic logging to drill into problems, etc. And, Borg was two generations ago! Even though I am very happy doing what I am now, sometimes I literally wake up in the morning thinking about Google's…

Eric Brewer is still on the faculty at Berkeley, just taking an extended leave at Google. He's scheduled to teach the undergrad database class next fall: http://www.eecs.berkeley.edu/Scheduling/CS/schedule-next.htm...

Re: Interview with Eric Brewer

#45

Post by mixmastamyk »

jacquesm wrote: post #30
mixmastamyk wrote: post #24

Earlier quoted context omitted.

The linux kernel knows what is happening in the containers. There is little performance overhead, and security is not a big advantage of the approach, VMs are better.

The linux kernel knows what's happening inside the containers but it can not de-duplicate any components that are present in multiple individual containers and it can not ensure that only one version of a package is present. This will potentially lead to cache trashing and more IO than strictly speaking required for a given workload. Of course it does make it easier to package and deploy applications (and to ensure t…

I'm not sure there will be enough duplication to make a difference most of the time. However, it will be less than with dedicated hardware.

There is also ksmd that is useful with VMs, where memory is at a premium, though I'm not certain it is compatible with lxc yet.

Re: Interview with Eric Brewer

#46

Post by falcolas »

DannoHung wrote: post #10
jacquesm wrote: post #9

Earlier quoted context omitted.

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Sorry, but what costs do containers incur? From my understanding, the resource overhead should be exceedingly minimal (disk space would ostensibly be the largest drawback, if you don't spend time cutting out the fat. Personally, I see this as a tooling issue since fat containers are completely orthogonal to how a container really executes). I understand some of the situation with IO isn't perfect yet, but I haven't h…

Containers, by themselves? Very little. Containers, as implemented by Docker, Rkt and Systemd? Quite a bit.

Disk - the technologies used for disk isolation (save chroots) are very poor performance, and in some cases can cause resource contention between what would otherwise appear to be unrelated containers. As an exmaple, using AUFS with Node creates a situation where any containers running on the same file system can only run one at a time, regardless of the number of cores. It's silly. Device mapper, on the other hand, is just plain slow (and buggy, when used on Ubuntu 14.4).

Network: The extra virtual interfaces, natting, and isolation all come with a performance penalty. For small payloads, this manifests as a few milliseconds of extra latency. For transferring large files, it can result in up to half of your throughput lost. Worse, if you have two docker containers side by side but due to your discovery mechanisms one container uses the host device to talk to the other container, you create what is known as assymetric TCP, which can cut your performance by a fifth or more. Try it out sometime, it's entertainingly frustrating to figure out.

Security: My favorite. What's the point of creating a container for your application if you're going to include the entire OS (and typically not even bother to update it with security patches). A real simple DOS on docker boxes would be to get the process to fill the "virtual" disk with cruft. You'll impact all running processes, the underlying OS (/var/lib/ is typically on the same device as /), and create such a singularly large file that it's usually easier to drop the entire thing and re-pull images instead of trying to trim it down.

Sorry if I sound down on the tech, but I've been fighting to make this work for production, and all of these little niggles are driving me batty.

Re: Interview with Eric Brewer

#47

Post by jacquesm »

menage wrote: post #31
jacquesm wrote: post #17

Earlier quoted context omitted.

> Primarily because dedicated servers are a lot less efficient Assuming you're operating at scale I don't see why that would be the case. And if you're not, what's the point? > The more different things you can pack on a machine while still ensuring that the high-priority/low-latency jobs get prompt access to the resources that they've reserved, the higher overall utilization you can achieve (and hence bring costs do…

Yes, I think it's safe to say that Google operates at scale. A lot of user-facing services at Google have to be over-provisioned in order to handle the cyclical usage patterns (the daily query peak is far higher than the average for most services) and to be able to survive the loss of a datacenter or two. This results in a lot of under-utilized servers for a big fraction of the time. So by packing lots of medium and…

I think it's a fairly safe assumption to say that Google (and FB and a bunch of other extremely large web properties) run into different problems than those that are faced on a day-to-day basis by most run-of-the-mill web companies.

Thank you for the insight into the number of processes inside a typical Borg container, so that was basically a kind of 'heavy process' rather than a complete application with all dependencies (including other processes the main one depended on) packaged in, this is something I wasn't expecting at all.

Re: Interview with Eric Brewer

#48

Post by jacquesm »

mixmastamyk wrote: post #45
jacquesm wrote: post #30

Earlier quoted context omitted.

The linux kernel knows what's happening inside the containers but it can not de-duplicate any components that are present in multiple individual containers and it can not ensure that only one version of a package is present. This will potentially lead to cache trashing and more IO than strictly speaking required for a given workload. Of course it does make it easier to package and deploy applications (and to ensure t…

I'm not sure there will be enough duplication to make a difference most of the time. However, it will be less than with dedicated hardware. There is also ksmd that is useful with VMs, where memory is at a premium, though I'm not certain it is compatible with lxc yet.

I can see how there will be less duplication than with virtualization (because you share the kernel, rather than running multiple instances of the kernel) but I can't see how duplication will be less than with dedicated hardware.

Re: Interview with Eric Brewer

#49

Post by nickpsecurity »

One thing that bothers me about the article is that it shows a recurring problem: IT not knowing what it knows. The NoSQL movement didn't notice that NonStop Architecture scaled linearly to thousands of cores with strong-consistency, five 9's, and SQL support. In the mid-80's. Instead of making a low-cost knockoff, like cluster movement did for NUMA's, they ditched consistency altogether and launched NoSQL movement. Now, I see man who invented CAP theorem discuss it while referencing all kinds of NoSQL options to show us the tradeoffs. Yet, there's Google services in production and tech such as FoundationDB doing strong consistency with distributed, high throughput and availability.

http://www.theregister.co.uk/2012/11/22/foundationdb_fear_of...

So, why aren't such techs mentioned in these discussions? I liked his explanation of the partitioning problem. Yet, he and NoSQL advocates seem unaware that numerous companies surmounted much of the problem with good design. We might turn CAP theorem into barely an issue if we can get the industry to put the amount of innovation into non-traditional, strong-consistency architectures as they did into weak-consistency architectures. There is hope: Google went from a famous, NoSQL player to inventing an amazing, strong-consistency RDBMS (F1). Let's hope more follow.

https://static.googleusercontent.com/media/research.google.c...

Re: Interview with Eric Brewer

#50

Post by mbesto »

jacquesm wrote: post #9
MCRed wrote: post #5

I'd like a real explanation for why containers are better than unikernels. Yes, unikernals are still early, and containers are convenient, because you have all of linux there... but it seems that running several linuxes on a linux machine is a bit much. One operating system plus XEN plus several applications in unikernels seems more efficient, and more exciting. But it's the less common choice. I am guessing convenie…

I'd like a real explanation for why containers and unikernels are better than regular run-of-the-mill applications running on dedicated servers. It's almost as if the wild west of the web isn't quite enough and we now need to add another explosion of layers-of-abstraction but this time on the server in order to pretend we have infinite hardware which then becomes it's own reason for existence rather than to simply ru…

Do you have experience using containers?

Here's my answer for "why": DRY. Once you've deployed hundreds of servers using the same exact Ubuntu 12.04 LTS kernel base, why not just completely abstract the OS away and focus the attention on scaling the OS services that matter? Why is that when I decide that I need to scale out, I need to copy every library of the OS and every line of code for the kernel and redeploy it every time I add a node?

> At this rate we'll end up shipping containers as 'apps' to the clients machines with a suitable emulator at some point.

That's exactly the point. Care to elaborate on the downside of such a promise?

Post reply on HN