Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making containers safer

lwn.net

Post reply on HN
Print view View original

21–30 of 52 posts

Re: Making containers safer

#21

Post by hardwaresofton »

scaryclam wrote: post #15
hardwaresofton wrote: post #9

LXC+LXD is one of the most undervalued container technologies out there. It can do a lot of cool things (like live migration via criu), and is IMO more production ready than Docker ever was. For those wondering about the distinction between "system" containers and "app" containers, the difference is user namespacing -- I think we should stop using the naming difference but instead go with specifying that the containe…

I'm curious as to how you get your images built. I'd love to remove the over reliance on docker in my team, but the sticking point seems to be building and storing images. I know that Kubernetes can run with different runtimes, and that docker images can be oci compliant, but it's a hard sell to ask everyone to try something different in production when they do everything else with docker. Do you have a workflow that…

I build my images for private & professional projects in Gitlab CI and I use the provided free registry.

The biggest downside to my set up is that I can't say I have enterprise-level security. Getting it nailed down properly requires the use of tools like TUF/Notary[0] & signed-image aware container repositories like Harbor[1] and a deployment gate mechanism like Portieris[2]. That's a lot of complexity to tack on.

Just a note -- Docker already runs containerd underneath via a shim[3]

If you're relying on docker-specific features then by all means it makes sense to continue using docker but if you're just looking for a thing to quietly run your containers (or power your kubernetes cluster), containerd should probably be that thing. It's all of the building and none of the extra stuff that docker the company is trying to do/become.

[0]: https://github.com/theupdateframework/notary

[1]: https://github.com/goharbor/harbor

[2]: https://github.com/IBM/portieris

[3]: https://groups.google.com/forum/#!topic/docker-dev/zaZFlvIx1...

Re: Making containers safer

#22

Post by dragonsh »

paulfurtado wrote: post #16
hardwaresofton wrote: post #8

Earlier quoted context omitted.

> (The main downside of hypervisors is that they are difficult to run on low-cost commodity cloud like Digital Ocean. You are forced to use bare metal stuff like Hetzner, AWS, Paket Cloud etc. The whole point of cloud is that the hardware/infrastructure is mostly abstracted away. If I have to run my own hypervisor just to ensure the container doesn't get broken out of, what's the point even calling it cloud?) Sorry i…

Kata works pretty well. You set it as a runtime for docker and it gives a fantastic illusion that the container isn't different from any other docker container but uses a KVM VM with minimal hardware to run the container. It starts in under 200ms (and faster if you make it use firecracker), etc. The problem your parent is pointing out is that AWS instances don't support nested virtualization (and it sounds like neith…

We tried kata the user experience is not as good as lxd.

Moreover with lxd just use the same deployment platform, scripts as used for bare-metal and VM. No need to fiddle with mix of shell scripts and DSL for container orchestration and how to integrate with your own code. No need for zombie process like in docker.

Re: Making containers safer

#23

Post by hardwaresofton »

wikibob wrote: post #18
hardwaresofton wrote: post #9

LXC+LXD is one of the most undervalued container technologies out there. It can do a lot of cool things (like live migration via criu), and is IMO more production ready than Docker ever was. For those wondering about the distinction between "system" containers and "app" containers, the difference is user namespacing -- I think we should stop using the naming difference but instead go with specifying that the containe…

I believe ContainerD is the default on Google Kubernetes Engine (GKE)

Didn't know this but this is great news -- last I heard it was experimental[0].

[0]: https://cloud.google.com/blog/products/containers-kubernetes...

Re: Making containers safer

#24

Post by trabant00 »

necovek wrote: post #14
trabant00 wrote: post #7

I used containers in production since 2002. We knew back then that containers are escapable and nothing changed since. Containers are for ease of management - package and configuration sepparation. One role per container. In absolutely no case are they meant for multi tenant cases. Containers are fine the way they are. Just use them for the right job.

So instead of trying to improve their security and keep the startup/overhead advantage over VMs in the orders of magnitude, you simply suggest doing nothing? FWIW, I like it that both camps are working on their shortcomings: LXD/container camp on isolation, and the VM camp on overhead (eg. firecracker).

I suggest using the right tool for the right job. There is no such thing as "shared but separate". After Spectre, Meltdown and co. everybody should understand this. It's not a limitation of current hardware/code, it's a limitation or principle.

Re: Making containers safer

#25

Post by hardwaresofton »

paulfurtado wrote: post #16
hardwaresofton wrote: post #8

Earlier quoted context omitted.

> (The main downside of hypervisors is that they are difficult to run on low-cost commodity cloud like Digital Ocean. You are forced to use bare metal stuff like Hetzner, AWS, Paket Cloud etc. The whole point of cloud is that the hardware/infrastructure is mostly abstracted away. If I have to run my own hypervisor just to ensure the container doesn't get broken out of, what's the point even calling it cloud?) Sorry i…

Kata works pretty well. You set it as a runtime for docker and it gives a fantastic illusion that the container isn't different from any other docker container but uses a KVM VM with minimal hardware to run the container. It starts in under 200ms (and faster if you make it use firecracker), etc. The problem your parent is pointing out is that AWS instances don't support nested virtualization (and it sounds like neith…

As a person who runs on (and really likes) Hetzner... I totally misunderstood, thanks for clearing it up.

Re: Making containers safer

#26

Post by paulfurtado »

dragonsh wrote: post #22
paulfurtado wrote: post #16

Earlier quoted context omitted.

Kata works pretty well. You set it as a runtime for docker and it gives a fantastic illusion that the container isn't different from any other docker container but uses a KVM VM with minimal hardware to run the container. It starts in under 200ms (and faster if you make it use firecracker), etc. The problem your parent is pointing out is that AWS instances don't support nested virtualization (and it sounds like neith…

We tried kata the user experience is not as good as lxd. Moreover with lxd just use the same deployment platform, scripts as used for bare-metal and VM. No need to fiddle with mix of shell scripts and DSL for container orchestration and how to integrate with your own code. No need for zombie process like in docker.

Kata is great if you're already using docker, it also is increased isolation versus lxd.

If you don't already have a use case for docker, kubernetes, the image repository features, etc then of course kata wouldn't be useful to you over lxc.

Also, in case you're dealing with zombie processes in docker again and can't ditch docker, running a container with `docker run --init` gets you a minimal init process that reaps the zombies. Adding that flag to dockerd instead does this for all containers by default. It's insane that this isn't turned on by default since it's such a common problem. Additionally, if you want to run systemd inside the container similar to lxc, you can install the systemd-oci-hook and it will do the necessary setup for systemd to be happy.

Re: Making containers safer

#27

Post by paulfurtado »

scaryclam wrote: post #15
hardwaresofton wrote: post #9

LXC+LXD is one of the most undervalued container technologies out there. It can do a lot of cool things (like live migration via criu), and is IMO more production ready than Docker ever was. For those wondering about the distinction between "system" containers and "app" containers, the difference is user namespacing -- I think we should stop using the naming difference but instead go with specifying that the containe…

I'm curious as to how you get your images built. I'd love to remove the over reliance on docker in my team, but the sticking point seems to be building and storing images. I know that Kubernetes can run with different runtimes, and that docker images can be oci compliant, but it's a hard sell to ask everyone to try something different in production when they do everything else with docker. Do you have a workflow that…

cri-o and containerd run standard docker images seamlessly. They're mostly drop-in replacements for dockerd. So even if you're using standard dockerd at build time, that's perfectly fine.

Things like buildah, kaniko, img, and whatnot can build standard Dockerfiles fine and push to standard docker registries. I don't have much experience with them though.

Re: Making containers safer

#28

Post by paulfurtado »

nisa wrote: post #13

The more I use Linux Containers (big fan of LXD) the more I'm convinced Solaris got it right with Zones, Crossbow, ZFS, SMF also for Zone services - Linux is still not there but instead we have lot's of vendor glue-code in go and many asteriks what is not possible. IMHO Linux should implement something like the zone concept that conceals cgroups, network, mount-namespaces - quota is still broken in btrfs, you can't d…

What issue did you face around network namespace awareness in this setup?

Re: Making containers safer

#29

Post by pjmlp »

nisa wrote: post #13

The more I use Linux Containers (big fan of LXD) the more I'm convinced Solaris got it right with Zones, Crossbow, ZFS, SMF also for Zone services - Linux is still not there but instead we have lot's of vendor glue-code in go and many asteriks what is not possible. IMHO Linux should implement something like the zone concept that conceals cgroups, network, mount-namespaces - quota is still broken in btrfs, you can't d…

HP-UX vaults were also quite good.

Re: Making containers safer

#30

Post by meddlepal »

paulfurtado wrote: post #16
hardwaresofton wrote: post #8

Earlier quoted context omitted.

> (The main downside of hypervisors is that they are difficult to run on low-cost commodity cloud like Digital Ocean. You are forced to use bare metal stuff like Hetzner, AWS, Paket Cloud etc. The whole point of cloud is that the hardware/infrastructure is mostly abstracted away. If I have to run my own hypervisor just to ensure the container doesn't get broken out of, what's the point even calling it cloud?) Sorry i…

Kata works pretty well. You set it as a runtime for docker and it gives a fantastic illusion that the container isn't different from any other docker container but uses a KVM VM with minimal hardware to run the container. It starts in under 200ms (and faster if you make it use firecracker), etc. The problem your parent is pointing out is that AWS instances don't support nested virtualization (and it sounds like neith…

Now that you mention that, it is kind of odd that AWS continues to not have nested virt considering both Azure and GCP both have it.
Post reply on HN