Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does Docker affect energy consumption?

arxiv.org

Post reply on HN
Print view View original

81–88 of 88 posts

Re: How does Docker affect energy consumption?

#81

Post by derefr »

sdeziel wrote: post #63
acdha wrote: post #46

Earlier quoted context omitted.

> Of course you are condensing. Those small VMs would have been running a full OS runtime each. That's something but probably not as much as you think since hypervisors can share identical pages (e.g. the Linux kernel) across guests and the base footprint for a server Linux install is not that high as a percentage of the private data most applications use. Unless you're running a ton of unnecessary services on those…

> since hypervisors can share identical pages (e.g. the Linux kernel) With ASLR, I'm now sure the gains are that substantial.

The parent doesn't mean that the hypervisor will merge identical MMU->physical page mappings (like a Copy-on-Write process fork would); they mean that VM pages' underlying host virtual pages literally get periodically hashed for their current content by a background process on the dom0 and merged when they are found to have identical hashes. The underlying virtual page is then made copy-on-write.

Or, to put that another way: the host memory for most modern hypervisors consists of a heap of "new" pages, and then a generational garbage collector that moves said pages, if still alive, into a content-addressible "old" store.

As such, if two VMs each have a process that

1. calls malloc() 1000 times to get 1000 1-page buffers randomly spaced through their memory, the mappings different for each VM; and then

2. uses a fixed PRNG seed to generate random data [but the same random data] to fill those pages;

then those two processes' pages will still get collapsed together for a 50% savings.

Re: How does Docker affect energy consumption?

#82

Post by derefr »

sdeziel wrote: post #64
chc wrote: post #56

Earlier quoted context omitted.

Doesn't LXC also mean containers? Like, the C literally stands for "Container."

LXC (with LXD) or OpenVZ containers are typically shipping full OS in the container. Docker is different in that it typically only have a few processes per containers.

"Full OS" is confusing here. Let's be explicit:

1. "application containers" are effectively a single process [though that can fork more] with some kernel process-struct fields set to nonzero values, indicating that the kernel should present this process a different view of its environment.

2. "virtual machines" are the processor providing a separate virtualized view of the CPU, on which is then booted another virtualized kernel, which brings up with it virtualized OS services and eventually an app.

3. Between them, "OS containers" are a hybrid: they start up all the userland virtualized OS services that a VM does, but they do so on top of a kernel that's not actually a fresh, separate kernel; but instead a kernel that has been told (through setting tons of containerization process flags) to present to this group of processes a view of the world where this kernel looks like a fresh kernel in a newly-started VM.

"OS containers" are basically a raw optimization over VMs by asking one kernel to pretend to be multiple kernels, and to manage one pool of memory instead of having multiple pools of memory. Anything you can do with raw VMs, you should (in theory, given good inter-container isolation+quota logic) be able to do with OS containers as well.

Re: How does Docker affect energy consumption?

#83

Post by derefr »

raulk wrote: post #43
AnsemWise wrote: post #39

Earlier quoted context omitted.

Containers are an abstraction that exist using cgroups and namespaces for isolation. They use the hosts' kernel, it's not virtualized. Containers are only limited by the capabilities of namespaces and cgroups, unlike vms. You might be dismissing microservices too quickly. They do have overhead but so does any level of abstract; the benefit of them though is clear separation of responsibilities between services and re…

Using VMs to isolate single processes is like owning multiple toasters, and buying a different house to plug each toaster in.

What if you're hosting toasters owned by people who might be rude teenagers who want to set any house with a rival toaster in it on fire?

When Docker can safely protect a Minecraft server in one container from a local DoS attack coming from a bot running in a sibling container, I'll reconsider using VMs. :P

Re: How does Docker affect energy consumption?

#84

Post by derefr »

infogulch wrote: post #54
naasking wrote: post #45

Earlier quoted context omitted.

> Containers allow us to condense workloads in a single OS runtime –while preserving isolation– where otherwise the same workloads would have spanned multiple machines or VMs, each with its overhead and slack (unused resources). Ultimately there's no reason why containerization can't be pushed right down to the language level. Consider .NET's AppDomains, or even further, a capability-secure programming language which…

> no reason why containerization can't be pushed right down to the language level I started my reply by listing a plethora of reasons why this wouldn't work. (For one, this would work because it does work, right now in Erlang.) But they all came down to that it seems like you're missing some of the problems that containerization solves. You can wrap up almost any service -- regardless of language or versions or runti…

> For one, this would work because it does work, right now in Erlang.

Eh? Erlang has no equivalent to cgroups/namespaces, or even an equivalent of non-UID-0 code execution on its VM. There is in fact no isolation mechanism in the Erlang VM; all code is "privileged." Untrusted multitenant code execution is a pipe-dream for now, unless you graft on another sandbox inside Erlang, ala CouchDB's V8 C-port [and more recently luerl] sandboxes.

(I've been very much considering contributing code for "non-privileged Erlang processes" and "Erlang process namespaces"—adding things like "namespace outboxes" that will crash their own virtual nodes rather than flood peers—but it's not there right now.)

Re: How does Docker affect energy consumption?

#85

Post by infogulch »

derefr wrote: post #84
infogulch wrote: post #54

Earlier quoted context omitted.

> no reason why containerization can't be pushed right down to the language level I started my reply by listing a plethora of reasons why this wouldn't work. (For one, this would work because it does work, right now in Erlang.) But they all came down to that it seems like you're missing some of the problems that containerization solves. You can wrap up almost any service -- regardless of language or versions or runti…

> For one, this would work because it does work, right now in Erlang. Eh? Erlang has no equivalent to cgroups/namespaces, or even an equivalent of non-UID-0 code execution on its VM. There is in fact no isolation mechanism in the Erlang VM; all code is "privileged." Untrusted multitenant code execution is a pipe-dream for now, unless you graft on another sandbox inside Erlang, ala CouchDB's V8 C-port [and more recent…

Well, no, it doesn't have privilege isolation. But it does have isolation from a fault tolerance perspective which I admit is only a portion of the purpose of isolation.

But if you're deliberately running malicious code even cgroups/namespaces won't save you from some attacks. Timing and cache attacks can be done without breaking out of the jail.

Re: How does Docker affect energy consumption?

#86

Post by derefr »

infogulch wrote: post #85
derefr wrote: post #84

Earlier quoted context omitted.

> For one, this would work because it does work, right now in Erlang. Eh? Erlang has no equivalent to cgroups/namespaces, or even an equivalent of non-UID-0 code execution on its VM. There is in fact no isolation mechanism in the Erlang VM; all code is "privileged." Untrusted multitenant code execution is a pipe-dream for now, unless you graft on another sandbox inside Erlang, ala CouchDB's V8 C-port [and more recent…

Well, no, it doesn't have privilege isolation. But it does have isolation from a fault tolerance perspective which I admit is only a portion of the purpose of isolation. But if you're deliberately running malicious code even cgroups/namespaces won't save you from some attacks. Timing and cache attacks can be done without breaking out of the jail.

Not deliberately running malicious code, no; but deliberately running user-supplied code, yes. A properly isolated container system (whether at the OS or the runtime level) allows one system to serve as host for the programmatic equivalent of members of two rival gangs (mutually-untrustworthy processes), without anyone "getting shot."

Re: How does Docker affect energy consumption?

#87

Post by infogulch »

derefr wrote: post #86
infogulch wrote: post #85

Earlier quoted context omitted.

Well, no, it doesn't have privilege isolation. But it does have isolation from a fault tolerance perspective which I admit is only a portion of the purpose of isolation. But if you're deliberately running malicious code even cgroups/namespaces won't save you from some attacks. Timing and cache attacks can be done without breaking out of the jail.

Not deliberately running malicious code, no; but deliberately running user-supplied code, yes. A properly isolated container system (whether at the OS or the runtime level) allows one system to serve as host for the programmatic equivalent of members of two rival gangs (mutually-untrustworthy processes), without anyone "getting shot."

Separating "malicious code" from "user-supplied code" is a distinction without a difference.

I would say one member having their private keys stolen[1] is a "fatal shot".

[1]: https://news.ycombinator.com/item?id=11891579

Re: How does Docker affect energy consumption?

#88

Post by Chris2048 »

CrLf wrote: post #37
raulk wrote: post #34

Doesn't this article fail to consider the bigger, practical picture, i.e. overall resource efficiency/footprint in multi-component architectures? Containers allow us to condense workloads in a single OS runtime –while preserving isolation– where otherwise the same workloads would have spanned multiple machines or VMs, each with its overhead and slack (unused resources). Example: consider you need to deploy not just a…

> Containers allow us to condense workloads in a single OS runtime –while preserving isolation– where otherwise the same workloads would have spanned multiple machines or VMs, each with its overhead and slack (unused resources). I'm starting to believe that the increased density provided by containerization is a myth, in practice. Both because orchestration tools bring their own overhead (compare all the proxies and…

But you can argue the overheads are birthing pains.
Post reply on HN