This was an awesome article.
It lays out every single hard lesson I discovered when working with docker compose.
Great minds think alike! (Or I just got lucky)
Docker Compose best practices for dev and prod
121–130 of 166 posts
Re: Docker Compose best practices for dev and prod
#122docker compose was one of the most spectacular and developer friendly deployment tool i have used. unfortunately the ecosystem is on kubernetes. you want to integrate with spot instance bidding on AWS...u need kubernetes. you want monitoring tools...u need kubernetes, etc the Compose spec is now open and standardised - https://www.compose-spec.io/ a kubernetes distro that can be managed entirely using compose files w…
When you say 'a kubernetes distro that can be managed entirely using compose files will be a massively impactful project. Not like Kompose which converts Compose files to k8s yml files....'
Do you mean from the your perspective, you set up your docker-compose and this startup ends up deploying pods into Kubernetes based on your compose file? Basically cutting out the middleman of you having to deal with Kubernetes yaml files?
Re: Docker Compose best practices for dev and prod
#123Earlier quoted context omitted.
what's the use case for backing up running containers? the code in the docker image is immutable (or should be), config goes into .env file and permanent data should go to a mounted volume in the host or an external database, so what do you want to backup?
ok, how do you backup your database volumes? when db is running.
same as if the database binary is running in the host? no difference with the db binary running in a container.
Re: Docker Compose best practices for dev and prod
#124docker compose is old hat, using podman to generate k8s Kubefiles for pods is where it's at
Re: Docker Compose best practices for dev and prod
#125Earlier quoted context omitted.
> ... if Dev and Prod run differently, there's really no point to ... "If dev and prod run differently" — Why is there an "if" here at all? Isn't it obvious to verify with two eyes in the real world? Is there any company in the world that runs production from somebody's laptop at home? Or distributes server racks for people to take home? It looks like an extremely disillusioned question out of touch with reality. 1.…
If the dev environments are on AWS with real S3 buckets (etc), and everything runs as an auto scaling group, is it not possible to use the same exact docker images (and terraform) in dev and prod? We haven't achieved it (yet?) but that's the direction I was aspiring to!
I was responding to the comment parent. Your article was practical and spot on.
Re: Docker Compose best practices for dev and prod
#126Earlier quoted context omitted.
Also, don't use Docker Compose in dev if you use Kubernetes in prod. Just use the same thing everywhere. Otherwise the fact that something works in dev tells you nothing about whether it would work in prod.
Running a local k8s cluster doesn't guarantee that a remote cluster on some cloud provider will work the same either. I've heard of and worked with multiple dev teams that develop with compose and deploy a totally different way (k8s, AWS auto scaling groups, heroku, etc), to great success. Devs know about compose and understand it. As long as the containers behave as planned, the DevOps people can figure out how to d…
That's all fine, until the apps that ran fine on a laptop won't run in the real environments, and then the devops people need to loop in the devs to debug it. Quite a lot of the time, it will be a problem with the manifest not providing something the app container expects, like a permission or a port binding or a missing ConfigMap. Everyone loses productivity when that happens.
Re: Docker Compose best practices for dev and prod
#127I tend to have a compose file that is designed to be used with `envsubst` for deployment via the CI, so a CI specific template. This way the CI can create the final file and `scp` it over to the host before calling `docker compose up`. The file always ends up with the same name and same location on the host so `docker compose` can detect any changes. So far haven't had any issues, but definitely not as clear as having `compose.dev.yml`/`compose.prod.yml` in source control, though very flexible.
Made a video of using this pattern with Blazor WASM, SQLite and Litestream here [1].
Re: Docker Compose best practices for dev and prod
#128huh weird, I still remember back then reading that docker-compose was not supposed to be used in production somewhere? However upon reading docker-compose official docs now they actually mention how to use in production.. of course because of that I had to learn kubernetes for months (no regret), however if docker-compose is suggested for production back then, I would have used it just for the speed of deployment
Re: Docker Compose best practices for dev and prod
#129Earlier quoted context omitted.
Everything is simple until someone asks you to try and diagnose (and hopefully fix) a bug on an upgrade between two different versions of a piece of software that can be deployed using 5 different strategies or that has 20 moving services...
the point of docker-compose is partly so that your stack is ephemeral? you stand things up and shut it down with one simple switch and it can move between different environments and use the same declaration as the last environment. so now I gotta drag your bash script around and embed my yaml file in it? lol
The point of docker compose is to orchestrate containers. It's great for you to have lines that you always color inside, but other people choose different lines.
Re: Docker Compose best practices for dev and prod
#130We've found https://tilt.dev/ to actually be a great alternative to docker-compose that also keeps the local setup and knowledge more inline with production (which is Kubernetes hosted)