Live data from Hacker News

Docker Compose Isn't Enough

blog.tealok.tech

111–120 of 208 posts

Re: Docker Compose Isn't Enough

#111
Docker Compose/SSH and some custom GitHub Actions to run DB Migrations is simple, straightforward and was more than enough to manage CI deployments all our Apps [1].

Although we're currently migrating to Kamal [2] to take advantage of its nicer remote setup, management and monitoring features.

[1] https://docs.servicestack.net/ssh-docker-compose-deploment

[2] https://kamal-deploy.org

Re: Docker Compose Isn't Enough

#112
post #80

Earlier quoted context omitted.

I'm using YAML despite it being garbage. (feels more common)

Maybe I am a boiled frog, but I personally like it. TOML looks fine for simple cases, but yaml is fine there as well. Also, yaml has very clean approaches for dealing with literal text. For instance, if your yaml file needs to include an xml file, a json file and a bash script, it all ends up being very readable.

I may consider using HOCON [0] if I see any traction, bust after writing a lot of YAML and even making my own YAML-driven tools, I feel its shortcomings are overstated. I got bit by corner cases maybe three or four times, and they didn't take long to debug.

[0] https://github.com/lightbend/config/blob/main/HOCON.md

Re: Docker Compose Isn't Enough

#113

Earlier quoted context omitted.

> For instance, you want to run one program that was written for Python 3.y, but also another program written for Python 3.z. You might be able to just install 3.z and have them both work, but it's not guaranteed. Worse, your OS version only comes with version 3.x and upgrading is painful. This is because the Linux model of global system wide shared dependencies is stupid, bad, and wrong. Docker and friends are a rou…

The Linux model works fine (very well, in fact, because of less HD space and much more importantly, less memory used for shared libraries) for programs that are normally included in the Linux distribution, since the whole thing is built together by the same organization as a cohesive whole. If every random little 20kB utility program were packaged with all its dependencies, the bloat would be massive. It doesn't work…

> because of less HD space and much more importantly, less memory used for shared libraries

Literally not in the Top 1000 problems for modern software.

> Windows went through a lot of similar problems, and had to go to great lengths to deal with it.

Not really. A 20 year old piece of windows software prettt much “just works”. Meanwhile it’s nigh impossible to compile a piece of Linux software that runs across every major distro in active use.

Re: Docker Compose Isn't Enough

#114
post #108

Just use k8s for prod. Docker compose is great for local dev, i would never dream of deploying to prod with compose.

Sure. But that’s not the point of the article. The articles point is “Docker Compose is too complicated”, then proposes a half baked implementation of a solution that hand waves away of all that complexity with a scenario that will fulfill their specific use case but will completely fall apart when it diverges from the specific way they’ve decided to build the abstraction layer.

The problem that the author refuses to accept is that deployment is inherently complex. It does require configuration of a lot of things because it supports every use case. So you either have a generic complex tool that everyone can use (like Docker Compose or Kube) or you have some specific tool that only works for a tiny subset of all users that is simpler that satisfies your use case.

Note that I’m not saying Docker Compose is perfect. The syntax is a bit arcane it’s complex to understand etc. But removing the complexity by removing configuration options is not the solution here. Instead the author should focus on different approaches and tools to manage the same existing level of abstraction.

And for what it’s worth, that’s essentially what Helm is for kube - a way to manage and hide away the complexity of kube manifests (but still use those manifests under the hood). But frankly, docker compose doesn’t need a helm. Because docker compose, as you point out, has value not as a deployment tool, but as a single file that developers can manage and spin up on their local machines in a manageable way that doesn’t have the author fighting YAML all day.

I would say if the author was actually interested in solving this problem in a productive way they should first try to see if docker itself is amenable to altering their constructs to provide optional higher abstractions over common concepts via the compose interface natively. If the source tools roll out those abstractions everyone will get them and adopt them.

Re: Docker Compose Isn't Enough

#115

You know you can run multiple processes inside a single container, right? The solution to the complexity of applications that are distributed as collections of interdependent containers is to put all of the different pieces inside a single container. This is equivalent to what people did before docker existed, and it's still perfectly viable today. There's no need to over-complicate things.

Is there a popular project which puts systemd and multiple processes in a container? I've seen only single process containers so far.

LXD/Incus for hypervisor-for-containers use-case.

Podman for "like Docker, but made by Linux people". It supports both application containers and system containers (which have systemd as PID 1).

Re: Docker Compose Isn't Enough

#116
post #85

Earlier quoted context omitted.

Sounds more like a skill issue. I am quite glad with traeffik and docker-compose. But I don't have that high loads or interactions on my servers

How many steps is it for you to add a new service to your system? Say you wanted to try out Paperless NGX, what would you need to do?

(Not parent) The quickest way I could do it is if I just used my existing wireguard setup to directly access the docker container IP. In that case it's just one compose up away even without mapping/exposing ports.

Re: Docker Compose Isn't Enough

#117
post #28

Earlier quoted context omitted.

it's perfectly fine if everything can run on a single server instance. which is probably the majority of things. i've run production instances with 2PB of data being scraped per month by 100+ CPUs and 256GB of RAM using docker compose. some of the machines (smaller instances) have run flawlessly with zero reboots for years on end. both on cloud and on-prem. i'd say its 100% production ready.

> I'd say its 100% production ready. That's fine. Some people also manually launch containers individually with Docker as their production system. They'd be wasting their time and making their lives needlessly harder, though. They already have tools that do all the legwork for them. Why not do the research and opt to force a square peg into a round hole? Research Docker swarm mode, reverse proxies like Traefik, and L…

Most people dont need rolling release, 24/7 availability, auto scaling, etc.. on their home server, so managing k8s just add way more complexity. My main reason to not use is because I would need to host a artifact service somewhere else which is PITA. Some k8s runtime support local building but is not as easy as compose in my experience.

Re: Docker Compose Isn't Enough

#118
post #108

Just use k8s for prod. Docker compose is great for local dev, i would never dream of deploying to prod with compose.

Just use Podman with plain simple .container file which is controlled entirenly by SystemD. Rock solid and easy to fix.

Re: Docker Compose Isn't Enough

#119
post #85

Earlier quoted context omitted.

Yup, I was surprised how weak the arguments were. I mean, surely there are better reasons why docker compose fails to scale? I have been pretty happy with it right now, and haven't felt the need to try something like k8s or docker swarm.

Sounds more like a skill issue. I am quite glad with traeffik and docker-compose. But I don't have that high loads or interactions on my servers

It is a skill issue, but not a problem. The developer who just wants to write unrelated code doesn't need Docker to write the code that goes into Docker.

For a lot of developers who have their primary focus set on writing code and documentation, trying to remember Docker or Kube on top of the rest of your stack is burdensome. Deployment is an end-user task. Obviously the development environment fits the target use-case already.

So please understand that there are lots of people who are building the things that go inside your containers who don't want, need, or see the value of these containers. For you they might offer value, but for someone like me they bring cognitive load and troubleshooting.

Too much abstraction can be just as bad as not enough. Like Akin's Laws of Spacecraft Design, but for software.

Re: Docker Compose Isn't Enough

#120

To those who are pointing out that compose isn't meant for production, keep in mind that this product that they're selling appears to be designed for the small time family self-hoster [0]. They're not targeting production web apps in a corporate setting, they seem to really be targeting people who wish that they could self-host something on their own network but don't have the technical background to use the docker c…

Yeah, we're very early building this, the blog post is just a way for me to organize my thoughts and start fights online. It's, uh, embarrassingly useful to yell semi-coherent thoughts into the void and have experts yell back with a decade or more of experience and information about tools I haven't heard of.

> I'm quite skeptical that adding a layer of abstraction and switching to TOML instead of YAML will suddenly enable those scared away by compose to start self-hosting, but kubernetes and docker swarm were never in the cards.

Yes, this is an excellent point. I did not articulate it well anywhere, but the goal is for users to have something more like Sandstorm, with a UI to install things. The TOML is for application developers, not end users. It'll either go in a separate database or, ideally, in the source code of the applications to be installed similar to a Dockerfile. I haven't started yet, but eventually we need to work with application developers to support things they want and to make it easier to treat Tealok as the "easy option" rather than docker compose.

Post reply on HN