Live data from Hacker News

Rootless Docker in a multi-user environment

cmtops.dev

11–20 of 50 posts

Re: Rootless Docker in a multi-user environment

#11
post #3

I wonder why people aren't using Singularity containers instead which seem to be rootless by design.

Oh man, Singularity… I once wrote: “In the shipping container analogy, you can think about Singularty containers as if they have no walls.” [1]

[1] https://sarusso.github.io/blog/container-engines-runtimes-or...

Re: Rootless Docker in a multi-user environment

#12
post #11
post #3

I wonder why people aren't using Singularity containers instead which seem to be rootless by design.

Oh man, Singularity… I once wrote: “In the shipping container analogy, you can think about Singularty containers as if they have no walls.” [1] [1] https://sarusso.github.io/blog/container-engines-runtimes-or...

Well, that blog post is getting a bookmark. Next time a junior dev asks me how this BS works, I'm going to show them your diagram.

Re: Rootless Docker in a multi-user environment

#13

I still don't get why multi user. I haven't seen a multi user scenario that makes any sense for any Linux deploy in ages. Is this a shared prod server? Is it a shared Dev server? Why would those be multi user? For prod, why isnt it cattle where you don't ever SSH onto the server? For Dev if you can't just run it all on your local machine why not do something like shipyard.build Somebody tell me what im missing.

Let's say you have some centralized monitoring of various host metrics. The ingestion process needs some amount of privileged access. But you don't want to give it full root. Meanwhile, your actual service probably needs no privileged access, save for some secrets which should also be inaccessible for the monitoring agent process. You may want to run both of these as containers.

I guess you may be using SELinux but even so, users and groups are natural parts of expressing and enforcing such constraints.

Re: Rootless Docker in a multi-user environment

#14
post #2

How well does rootless docker work compared to rootless podman?

If it works as I understood, in this setup I can see an advantage at an architectural level: in Podman containers images are stored on a per-user basis, while in this setup they would be shared between users, thus using much less disk space (if using the same base images). Besides this, I actually have the same question.

Re: Rootless Docker in a multi-user environment

#15
post #11
post #3

I wonder why people aren't using Singularity containers instead which seem to be rootless by design.

Oh man, Singularity… I once wrote: “In the shipping container analogy, you can think about Singularty containers as if they have no walls.” [1] [1] https://sarusso.github.io/blog/container-engines-runtimes-or...

Well, I'm glad I asked, appreciate your response and great blog post.

Re: Rootless Docker in a multi-user environment

#16
post #13

I still don't get why multi user. I haven't seen a multi user scenario that makes any sense for any Linux deploy in ages. Is this a shared prod server? Is it a shared Dev server? Why would those be multi user? For prod, why isnt it cattle where you don't ever SSH onto the server? For Dev if you can't just run it all on your local machine why not do something like shipyard.build Somebody tell me what im missing.

Let's say you have some centralized monitoring of various host metrics. The ingestion process needs some amount of privileged access. But you don't want to give it full root. Meanwhile, your actual service probably needs no privileged access, save for some secrets which should also be inaccessible for the monitoring agent process. You may want to run both of these as containers. I guess you may be using SELinux but e…

IMO, Kubernetes is going to be easier for an end user and more performant than rolling your own rootless container platform.

Re: Rootless Docker in a multi-user environment

#17

I still don't get why multi user. I haven't seen a multi user scenario that makes any sense for any Linux deploy in ages. Is this a shared prod server? Is it a shared Dev server? Why would those be multi user? For prod, why isnt it cattle where you don't ever SSH onto the server? For Dev if you can't just run it all on your local machine why not do something like shipyard.build Somebody tell me what im missing.

Aren't most HPC clusters multiuser?

Re: Rootless Docker in a multi-user environment

#18
post #3

I wonder why people aren't using Singularity containers instead which seem to be rootless by design.

At the risk of showcasing my bubble: Lack of exposure; how many people even know what singularity is or how to use it? I know it's used in scientific HPC, but I don't see evidence of wider adoption.

Definitely used in HPC environments where multi user is prevalent, I know of it because I used to admin a small HPC

Re: Rootless Docker in a multi-user environment

#19

I still don't get why multi user. I haven't seen a multi user scenario that makes any sense for any Linux deploy in ages. Is this a shared prod server? Is it a shared Dev server? Why would those be multi user? For prod, why isnt it cattle where you don't ever SSH onto the server? For Dev if you can't just run it all on your local machine why not do something like shipyard.build Somebody tell me what im missing.

>> Why would those be multi user? For prod, why isnt it cattle where you don't ever SSH onto the server?

I hate the cattle not pets analogy, do you know how well taken care of most cattle are?

If your operating at google/fb/apple scale then yes you can have this approach. There are lots of business that DON't need, want or have to scale to this level. There are lots of systems where this approach breaks down.

Docker is great if you're deploying some sort of mess of software. Your average node/python/ruby setup with its run time and packages + custom code makes sense to get shoved into a container and deployed. Docker turns these sorts of deployments into App Store like experiences.

It doesn't mean is the only, or a good approach... if your dev env looks like your production env, if your deploying binaries then pushing out as a c-group or even an old school "user" is not only viable it has a lot less complexity.

As a bonus you can SSH in, you have all the tools available on a machine, you can tweak a box to add in perf/monitoring/debug/testing tools on there to get to the source of those "it only happens here" bugs...

Re: Rootless Docker in a multi-user environment

#20
Here's our experience with rootless docker.

We've used it in a single-user (a docker user) and multi-user (user for each dev) environment.

Most, if not all, containers work fine, there are some, like mailcow which don't work well with it.

If you have multiple IPs on the one machine, there is a longstanding bug that means you can't bind the same port on different IPs. Eg IP1:80 and IP2:80. The workaround for this is separate rootless docker users + runtime for each container that shares ports, nasty.

In a multi-user environment we simply setup rootless docker under each devs user, so they have their own runtime and their own containers isolated from other devs. This works really well.

Overall it works well for us.

Post reply on HN