Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker Compose best practices for dev and prod

prod.releasehub.com

Post reply on HN
Print view View original

91–100 of 166 posts

Re: Docker Compose best practices for dev and prod

#91

Post by solardev »

user3939382 wrote: post #89
solardev wrote: post #48

If you happen to be unlucky enough to need Docker for LAMP/LEMP stacks, Lando is a neat open-source util one level of abstraction above Docker Compose: https://lando.dev/ Like with `lando init` and then `lando start`, it'll fetch a LEMP "recipe" (say, for Wordpress or Drupal), configure all the Docker containers for you, set up all the networking between them, get PHP xdebug and such working automatically, and leave…

I was kind of puzzled by your preamble: > If you happen to be unlucky enough to need Docker for LAMP/LEMP stacks We run Docker LAMP and have a new version underway that's LEMP, haven't run into any problems. xdebug works fine, I even got it working recently in ECS. I'm not sure what's different about LAMP/LEMP in Docker vs any other stack.

Sorry for being unclear. It's not that Docker doesn't work well with LEMP, it's that Lando in particular was designed around older stacks (LEMP/LAMP and caches) and wouldn't be appropriate if you were trying to spin up, say, Next.js or Deno or Cloudflare Workers or React Native stacks or whatever.

The "unlucky" part is just having to work with a LEMP stack at all, and that's just my personal bias leaking through my post, sorry. Having grown up with that stuff and used it until just last year, I am so so grateful that I was able to finally move into a frontend job where I don't have to manage the stack anymore. A new generation of abstracted backend vendors (headless CMSes coupled with Jamstack hosts) makes it so that there is a sub-industry of web devs who never have to touch VMs directly anymore. I, for one, couldn't be happier about that.

(But of course there will always be other use cases that require a fuller/closer to the metal stack, and also backend and ops people who love that work. I don't fault them in the least, I greatly respect them, I'm just glad I don't have to do that.)

Re: Docker Compose best practices for dev and prod

#92

Post by edfletcher_t137 »

adamckay wrote: post #88
pcorsaro wrote: post #77

Earlier quoted context omitted.

Just curious, why would a bind-mount be bad in production? I understand if you're running multiple app servers and need the code to be deployed in a lot of places that they'd be bad, but if you're just using a single server, what's the downside of using bind-mounts? You could use git to push code changes up to your server, and they'd be picked up immediately without having to rebuild and push an image.

Usually for production you want to know with 100% certainty what version of the software is running. Whilst you can do that with Git commit hashes you can't be sure no one has performed a hotfix and modified code on the production server which is not committed to source control. Then there's also the potential problem of pulling the latest updates from source control but forgetting to restart the container so it's no…

This, 100%. Reproducibility, traceability and auditability all go out the window when you allow source modification on in production directly. You never want to ask "what's running in production" only to be greeted by crickets in response, or worse a lone "my branch from five days ago."

Re: Docker Compose best practices for dev and prod

#93

Post by Ajedi32 »

I like to divide up my compose files into three:

1. A `docker-compose.yaml` for basic configuration common among all environments

2. A `docker-compose.dev.yaml` for configuration overrides specific to development

3. A `docker-compose.prod.yaml` for configuration overrides specific to production

Then I have `COMPOSE_FILE=docker-compose.yaml:docker-compose.dev.yaml` in my `.env` file in the root of the project so I can just run `docker compose up` in dev, and `COMPOSE_FILE=docker-compose.yaml:docker-compose.prod.yaml` in a `prod.env` file so I can run `docker compose --env-file=prod.env up` in production. Sometimes also a `staging.env` file which is identical to `prod.env` but with a few different environment variables.

It's been working pretty well for me so far.

Re: Docker Compose best practices for dev and prod

#94

Post by Ajedi32 »

bdcravens wrote: post #18

Surprised it didn't mention profiles, which let you launch only part of your "stack" (think background processing in addition to web app, etc) https://docs.docker.com/compose/profiles/ Prior to that feature, I had a hand-rolled approach using bash scripts and using overrides. Worked well enough, but felt like a dirty dependency.

One thing I really like about profiles is that they get activated implicitly if you reference one of their services when running docker compose. So you can define development, debugging, or maintenance tools in your compose file and just run them with `docker compose run some-random-tool` without having to worry about them interfering with the standard `docker compose up` workflow.

Re: Docker Compose best practices for dev and prod

#95

Post by Mavvie »

jaimehrubiks wrote: post #61

Do people here set CPU limits? (in either docker or kube) In my company we only set CPU requests + Memory req+limits. I read somewhere that CPU limits are kinda broken and are not reliable. In some experiments CPU limits tend to cause unnecessary throttling, which is why some people don't recommend them. However, I've also found some cases where I can't make the app to use only some of the CPUs, some frameworks tend…

In our k8s clusters we generally set both requests and limits on CPU and memory. We haven't noticed many problems, except for when some apps didn't even have CPU requests and dominated all the CPU on a cluster (which didn't even autoscale because the scheduler doesn't know about the problem).

I think CPU limits (and also, choosing to set memory limits equal to the requests or higher) depends on your goal. If you want to get the best use out of your hardware, set low requests and high limits and you'll be able to minimize "wasted" scaling. But if your goal is consistent performance and availability, setting high requests and high limits (usually equal to each other) save you from a whole host of problems.

Just my take. I have a fair bit of experience but am far from an expert on the topic. And my experience is only on the scale of dozens of applications/dozens of nodes...people with higher scale experience may feel differently.

Re: Docker Compose best practices for dev and prod

#96

Post by zoomzoom »

0xbadcafebee wrote: post #30

If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to troubleshoot two separate sets of problems, and testing being unreliable, defeating the whole "it just runs everywhere" premise. Docker Compose is fine for running smoke tests/unit tests, but if Dev and Prod run differently, there's really no point to using Do…

Totally agree that in practice you often need several configurations for the same systems across different environment types. For example, you probably also need a different docker-compose for running in CI when compared to dev or a deployed environment. The same applies for k8s and terraform, and many others.

Managing all these different configurations and the matrix of interactions between them is why we're trying to integrate the SDLC into one configuration at Coherence (withcoherence.com). [disclosure... I'm a cofounder]

Re: Docker Compose best practices for dev and prod

#97

Post by xorcist »

Ajedi32 wrote: post #93

I like to divide up my compose files into three: 1. A `docker-compose.yaml` for basic configuration common among all environments 2. A `docker-compose.dev.yaml` for configuration overrides specific to development 3. A `docker-compose.prod.yaml` for configuration overrides specific to production Then I have `COMPOSE_FILE=docker-compose.yaml:docker-compose.dev.yaml` in my `.env` file in the root of the project so I can…

In my experience, the "base" configuration should always be in use somewhere. Otherwise you inevitably end up with cruft, when default values turns out to have no practical value.

In your example, "dev" should probably be the default configuration, which "prod" overrides. It's a tiny detail but I think it pays over time.

A similar situation is when you have lots of almost-identical settings between many (micro)services.

Re: Docker Compose best practices for dev and prod

#98

Post by glintik »

lazyant wrote: post #63
glintik wrote: post #19

Wondered that author didn’t touch such important thing as backups of running containers.

what's the use case for backing up running containers? the code in the docker image is immutable (or should be), config goes into .env file and permanent data should go to a mounted volume in the host or an external database, so what do you want to backup?

ok, how do you backup your database volumes? when db is running.

Re: Docker Compose best practices for dev and prod

#99

Post by sureglymop »

biggerChris wrote: post #3

Docker is cool and all. Kids these days don’t know anything about Tmux and port 433. Exposing localhost to the internet and getting your internet monitored by your ISP and getting letters in the mail; for hosting movies and Storage services. Lol.

Am kid, do know

Re: Docker Compose best practices for dev and prod

#100

Post by lazide »

xorcist wrote: post #97
Ajedi32 wrote: post #93

I like to divide up my compose files into three: 1. A `docker-compose.yaml` for basic configuration common among all environments 2. A `docker-compose.dev.yaml` for configuration overrides specific to development 3. A `docker-compose.prod.yaml` for configuration overrides specific to production Then I have `COMPOSE_FILE=docker-compose.yaml:docker-compose.dev.yaml` in my `.env` file in the root of the project so I can…

In my experience, the "base" configuration should always be in use somewhere. Otherwise you inevitably end up with cruft, when default values turns out to have no practical value. In your example, "dev" should probably be the default configuration, which "prod" overrides. It's a tiny detail but I think it pays over time. A similar situation is when you have lots of almost-identical settings between many (micro)servic…

100% agree - I find staging useful here. It’s the default ‘non-production’, and has realistic data shared in a way that anyone in dev can access it. Think ‘sacrificial prod with no actual real customers hitting it right now’. Depending on data sensitivity requirements, it might have a subset of real data, or no real data.

Having prod be the default is a problem, as it’s inevitable someone will accidentally break production/write junk to it when they didn’t realize they were talking to prod.

Having dev in it inevitably results in broken prod pushes because no one has ever tried it outside of a synthetic dev environment.

Post reply on HN