Docker Compose Isn't Enough • Hacker News Board

Re: Docker Compose Isn't Enough

#171

Post by mkarrmann » Wed, Nov 13, 2024, 9:35 PM UTC

eliribble wrote: post #140

mkarrmann wrote: post #54

I'm pretty confused by this article. It says docker compose is at the "wrong-level of abstraction", but I kept feeling the author was instead expecting docker compose to solve different problems that it was ever meant to solve. In fact, they seem to be expecting a highly-opinionated, high-level interface which solves problems that I don't think anyone using docker compose in prod should even be worried about. A lot o…

These are great points, and probably worth their own blog post to answer. > First of all, why is this a concern? Idle threads have basically no impact on a system, so this generally isn't a concern Idle threads have very low impact on CPU utilization, probably, if the application is well-behaved (and I expect most databases and caching layers to be well-behaved in this way). The application itself, however, will need…

Ty, all good points

Re: Docker Compose Isn't Enough

#172

Post by orf » Wed, Nov 13, 2024, 9:52 PM UTC

anonzzzies wrote: post #154

dankle wrote: post #108

Just use k8s for prod. Docker compose is great for local dev, i would never dream of deploying to prod with compose.

We run large, auto scaling clusters with compose and a small orchestration thing we have been using since the 90s (written in perl) (before compose we had our own compose-like with chroot). No issues. For decades.

Cool - everyone knows it’s possible to hack together something, or anything else. But Is it a good idea to spend time and resources starting to do that now if you haven’t been doing it with a sketchy Perl script since the 90s? Not really.

Re: Docker Compose Isn't Enough

#173

Post by morcus » Wed, Nov 13, 2024, 9:53 PM UTC

SomeCallMeTim wrote: post #25

zemo wrote: post #20

people dramatically overestimate how difficult it is to write a program that controls docker for you. This is one of those things where you can write like two pages of Python and ignore... all this: > Tealok is a runtime we’re building for running containers. If you have one machine and docker-compose is falling short, really, just write a Python script with the official docker Python package, you'll be fine.

Ha. Just did almost exactly that, but with a Go script--I wanted my Docker Compose to auto-update when I built on my CI server. I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a…

> I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a tiny Go server that, when pinged with a shared secret, would cause it to run `docker compose up -d --pull always`.

Is the code for this public? I had the same desire when setting up the server for my personal project but as you mentioned I eventually decided it was OK to just poll every two minutes and I would rather work on the project than on configuring docker. What I would like though is cleaning up the older images that are no longer needed.

Re: Docker Compose Isn't Enough

#174

Post by orf » Wed, Nov 13, 2024, 10:06 PM UTC

stryan wrote: post #161

ianburrell wrote: post #158

Earlier quoted context omitted.

What happens when your machine dies? You get paged, your service is down, you have to migrate the services to a new machine. With Kubernetes, it starts new container on new machine, and you don't get paged. You could write scripts to do failover, but that takes works and will be buggy.

I was more questioning why a .container file would work for system services versus application services, since basically all those same problems occur for system services too. Either way this type of argument just comes down to "should I cluster or not" but to think out loud for a bit that's just basic HA planning: Simplest solution is keepalived/etc for stateful services, standard load balancing for stateless. Don't…

What’s the advantage of configuring, testing and maintaining all of that instead of a 10 line Kubernetes manifest that does more?

Re: Docker Compose Isn't Enough

#175

Post by datadeft » Wed, Nov 13, 2024, 10:37 PM UTC

eliribble wrote: post #136

datadeft wrote: post #127

In summary: Accidental complexity, accidental complexity, accidental complexity and when we use a tool that is designed for problem A for a problem B it is not sufficiently dealing with complexity. This is why we need a new tool. ¯\_(ツ)_/¯

While a bit of a hot take, you're not wrong. We need something that's less scalability focused than Kubernetes/Mesos/Docker Swarm but that doesn't put too much burden on application developers. Something that focuses on being secure, reliable, and understandable, in that order. I'm not aware of anything going for that niche. That means a new tool is in order.

I think we need a composable system but I am not sure if the current frame where this problem and these tools exist is good enough. We might need to rethink how we handle access and usage patterns well. I only have wrong answers. Docker compose is amazing for local dev env, k8s is terrible for production. These are my experiences with this domain.

Re: Docker Compose Isn't Enough

#176

Post by stryan » Wed, Nov 13, 2024, 11:01 PM UTC

orf wrote: post #174

stryan wrote: post #161

Earlier quoted context omitted.

I was more questioning why a .container file would work for system services versus application services, since basically all those same problems occur for system services too. Either way this type of argument just comes down to "should I cluster or not" but to think out loud for a bit that's just basic HA planning: Simplest solution is keepalived/etc for stateful services, standard load balancing for stateless. Don't…

What’s the advantage of configuring, testing and maintaining all of that instead of a 10 line Kubernetes manifest that does more?

Not having to install and manage Kubernetes? Unless you're paying someone else to run it for you (in which case this entire conversation is sort of moot as that's way out of scope for comparison) that stuff is all still running somewhere and you have to configure it. e.g. even in small-scale setups like k3s you have to set up shared storage or kube-vip yourself for true high availability. It's not some magic bullet for getting out of all operational planning.

Also even in separate components it's not really "all of that". Assume an example setup where the application is a container with a volume on an NFS share for state: on a given node we'd need to install podman and keepalived, a .container and .volume file, and the keepalived conf. An average keepalived conf file is probably 20ish lines long (including notification settings for failover, so drop like 8 lines if you don't care about that or monitor externally) and looking at an application I have deployed a similar .container file is 24 lines (including whitespace and normal systemd unit file boilerplate) and the NFS .volume file is 5 lines. So ballpark 50 lines of config, if you or others wanted to compare configuration complexity.

Also, fun fact, you could still even use that Kubernetes manifest. Podman accepts .kube files to manage resources against itself or a Kubernetes cluster; I've been recommending it as sort of a middle ground between going all in on k8s versus transitioning slowly from a non-clustered deployment.

Re: Docker Compose Isn't Enough

#177

Post by thecrash » Wed, Nov 13, 2024, 11:20 PM UTC

eliribble wrote: post #103

chipdart wrote: post #69

Earlier quoted context omitted.

> For many people self-hosting implies a personal server; it's not really "development" but it's not "production" either. There's Docker swarm mode for that. It supports clustering too. It's nuts how people look at a developer tool designed to quickly launch a preconfigured set of containers and think it's reasonable to use it to launch production services. It's even more baffling how anyone looks at a container orch…

Hey, blog post author here. I'm curious how that last sentence was going to end. Let's say I agree with you and that TLS termination is not a container orchestration responsibility. Where does the responsibility of container orchestration start and TLS termination end? Many applications need to create URLs that point to themselves so they have to have a notion of the domain they are being served under. There has to b…

If you want an HTTPS ingress controller that's simple, opinionated, but still flexible enough to handle most use cases, I've enjoyed this one: https://github.com/SteveLTN/https-portal

Re: Docker Compose Isn't Enough

#178

Post by selcuka » Wed, Nov 13, 2024, 11:52 PM UTC

chefandy wrote: post #27

zemo wrote: post #20

people dramatically overestimate how difficult it is to write a program that controls docker for you. This is one of those things where you can write like two pages of Python and ignore... all this: > Tealok is a runtime we’re building for running containers. If you have one machine and docker-compose is falling short, really, just write a Python script with the official docker Python package, you'll be fine.

I'm not dev-ops-familiar with Docker, so you might be more familiar with the problem space, but it seems like "You can just write a Python script to do what you want" is the sort of thing people say to justify not giving people the config infrastructure they need to solve their problems without having to write executable code. Like, sure, you often can make up for not having a zoom lens by walking towards what you're…

> you often can make up for not having a zoom lens by walking towards what you're photographing

Nitpicking: You can't really make up for not having a zoom lens as your field of view will stay the same. Hitchcock's famous "dolly zoom" demonstrates that quite nicely.

Re: Docker Compose Isn't Enough

#179

Post by shiroiushi » Wed, Nov 13, 2024, 11:56 PM UTC

forrestthewoods wrote: post #113

shiroiushi wrote: post #87

Earlier quoted context omitted.

The Linux model works fine (very well, in fact, because of less HD space and much more importantly, less memory used for shared libraries) for programs that are normally included in the Linux distribution, since the whole thing is built together by the same organization as a cohesive whole. If every random little 20kB utility program were packaged with all its dependencies, the bloat would be massive. It doesn't work…

> because of less HD space and much more importantly, less memory used for shared libraries Literally not in the Top 1000 problems for modern software. > Windows went through a lot of similar problems, and had to go to great lengths to deal with it. Not really. A 20 year old piece of windows software prettt much “just works”. Meanwhile it’s nigh impossible to compile a piece of Linux software that runs across every m…

>A 20 year old piece of windows software prettt much “just works”

No, it only works because Windows basically included something much like WINE (they call it WoW) in Windows, so old pieces of software aren't running on the modern libraries.

>it’s nigh impossible to compile a piece of Linux software that runs across every major distro in active use.

Sure you can, with Docker. It's effectively doing the same thing Windows does with WoW. Windows just makes it a lot more invisible to the user.

Re: Docker Compose Isn't Enough

#180

Post by kentonv » Wed, Nov 13, 2024, 11:58 PM UTC

eliribble wrote: post #150

kentonv wrote: post #145

Earlier quoted context omitted.

> I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Oh hai. Honestly I'm not sure I'm a reliable source for why we failed. It's tempting to convince myself of convenient excuses. But I really don't think the problem was with the idea. We actually had a lot of user excitement around the product. I think we screwed up the business strategy. We were too eager to generate revenue to…

Oh, hey, holy shit! You're one of my heroes. I've read through your internal discussions on protobuf within Google, you did amazing work there, and held your own against a difficult political environment. It sounds like you have no criticism of the technical approach, then, but rather just the business mechanics? That's eye-opening, given how much has changed in self-hosted deployment since Sandstorm started. If you…

Oh, well, thank you!

You should probably take me with a grain of salt: of course, I, the technical architect of Sandstorm, still think the technical architecture is great. ;) Whereas I never claimed to be good at business so I'm not adverse to reporting I am bad at business.

But yeah I still think it's the right architecture. Still believe in what's written here: https://sandstorm.io/how-it-works

But I do think there is a lot of work needed to get to a baseline of functionality that people will use. Bootstrapping a new platform is hard and needs a long runway. Could you find investors willing to back it? I don't know. I hated fundraising, though.

Of course, there are many lower-level details I would change, like:

* Consider using V8 isolates instead of containers. Sandstorm had a big problem with slow cold starts and high resource usage which V8 isolates could do much better with. It would, of course, make porting existing apps much harder, but ports to Sandstorm were always pretty janky... maybe would have been better to focus on building new apps that target Sandstorm from the start.

* Should never have used Mongo to store platform metadata... should have been SQLite!

* The shell UI should have been an app itself.

* We should never have built Blackrock (the "scalable" version of Sandstorm that was the basis for the Oasis hosting service). Or at least, it should have come much later. Should have focused instead on making it really easy to deploy to many different VPSes and federate per-user instances.