Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker Compose best practices for dev and prod

prod.releasehub.com

Post reply on HN
Print view View original

61–70 of 166 posts

Re: Docker Compose best practices for dev and prod

#61

Post by jaimehrubiks »

Do people here set CPU limits? (in either docker or kube)

In my company we only set CPU requests + Memory req+limits. I read somewhere that CPU limits are kinda broken and are not reliable. In some experiments CPU limits tend to cause unnecessary throttling, which is why some people don't recommend them.

However, I've also found some cases where I can't make the app to use only some of the CPUs, some frameworks tend to read the number of total available CPUs on the node and try to use them.

Re: Docker Compose best practices for dev and prod

#62

Post by JadoJodo »

One of the things we've really struggled with is sharing dev compose files between Windows and macOS. We were unable to make bind mounts work correctly with Windows, due to performance issues (this used to be documented in the Docker docs, but I can't seem to locate it now). This resulted in us falling back to VSCode's remote container tooling.

Has anyone else had this issue with Windows? The last time I looked into the issue was about 8 months ago, so it's possible the issues have been addressed since then.

Re: Docker Compose best practices for dev and prod

#63

Post by lazyant »

glintik wrote: post #19

Wondered that author didn’t touch such important thing as backups of running containers.

what's the use case for backing up running containers? the code in the docker image is immutable (or should be), config goes into .env file and permanent data should go to a mounted volume in the host or an external database, so what do you want to backup?

Re: Docker Compose best practices for dev and prod

#64

Post by tjoff »

dijksterhuis wrote: post #57
tjoff wrote: post #38

Earlier quoted context omitted.

That is an absurdly black-and-white view.

I have lost count of the times I've heard "but it works fine on MY machine, it must be the CI/CD that's broken" I then find out that no-one bothered to run the compose file that matches what gets put into prod on their local machines. My point is: your view might become more black and white on this matter after the N-th "but it works fine on MY machine" comment. EDIT: Where N = your personal tolerance level of bullsh…

That might me your experience, I don't share it.

But even so, how you can go from that to:

> [...] there's really no point to using Docker Compose other than developing without internet access

boggles my mind.

EDIT: apologies, mixup on my end and quote is from someone else.

Re: Docker Compose best practices for dev and prod

#65

Post by Gordonjcp »

switchbak wrote: post #40
CincinnatiMan wrote: post #15

I recently learned that moving the compose file around and running commands on it with it in different directories puts things into a bad state. This is because it apparently tags images and containers with context on where the compose file was run from. However, you can negate this by always passing in a custom project name with -p.

There's something about that decision that violates my mental model. Feels similar to a referential transparency issue: the state of the system should be the same regardless of the name of the directory something lives in.

You can specify a project name, but giving things rigidly-defined names breaks the whole idea of it.

Why would you want two different instances of the same project to have the same name?

Re: Docker Compose best practices for dev and prod

#66

Post by dijksterhuis »

0xbadcafebee wrote: post #30

If you don't use the same Docker Compose file for Production, don't use it for Development. Your two different systems will diverge in behavior, leading you to troubleshoot two separate sets of problems, and testing being unreliable, defeating the whole "it just runs everywhere" premise. Docker Compose is fine for running smoke tests/unit tests, but if Dev and Prod run differently, there's really no point to using Do…

> defeating the whole "it just runs everywhere" premise.

Straight up 100% hard agree.

> It goes without saying that Docker Compose only works on a single host, so that won't work for Prod if you need more than one host, but single-host-everything seems to be HN's current fetish)

technically this is only correct for compose file versions up to and including 3.8.

Part of the push with the new mainline compose V2 plugin has included the updated compose specification [0] which seems to have unify quite a lot of differences between the "compose" file for compose deployments and the "stack" file for swarm deployments.

FYI You can switch over to the new version with the compose V2 plugin by remvoing any `version` key from your compose file.

Although the `deploy: restart_policy:` values are currently not unified, much to my sadness.

[0]: https://docs.docker.com/compose/compose-file/

Re: Docker Compose best practices for dev and prod

#67

Post by pbalau »

switchbak wrote: post #40
CincinnatiMan wrote: post #15

I recently learned that moving the compose file around and running commands on it with it in different directories puts things into a bad state. This is because it apparently tags images and containers with context on where the compose file was run from. However, you can negate this by always passing in a custom project name with -p.

There's something about that decision that violates my mental model. Feels similar to a referential transparency issue: the state of the system should be the same regardless of the name of the directory something lives in.

Which is true if you pass -p. The current directory is merely the default.

Re: Docker Compose best practices for dev and prod

#68

Post by dijksterhuis »

tjoff wrote: post #64
dijksterhuis wrote: post #57

Earlier quoted context omitted.

I have lost count of the times I've heard "but it works fine on MY machine, it must be the CI/CD that's broken" I then find out that no-one bothered to run the compose file that matches what gets put into prod on their local machines. My point is: your view might become more black and white on this matter after the N-th "but it works fine on MY machine" comment. EDIT: Where N = your personal tolerance level of bullsh…

That might me your experience, I don't share it. But even so, how you can go from that to: > [...] there's really no point to using Docker Compose other than developing without internet access boggles my mind. EDIT: apologies, mixup on my end and quote is from someone else.

Not the parent! Made no comment on developing without internet access.

EDIT: I usually try to build, test, inspect, push and deploy from the same compose file. You can deploy to ECS direct from compose using an AWS context: https://aws.amazon.com/blogs/containers/deploy-applications-...

Re: Docker Compose best practices for dev and prod

#69

Post by tmpz22 »

JadoJodo wrote: post #62

One of the things we've really struggled with is sharing dev compose files between Windows and macOS. We were unable to make bind mounts work correctly with Windows, due to performance issues (this used to be documented in the Docker docs, but I can't seem to locate it now). This resulted in us falling back to VSCode's remote container tooling. Has anyone else had this issue with Windows? The last time I looked into…

You're trying to create a uniform developer experience across two platforms that are wildly different.

If you find yourself spending a ton of time on a one-size-fits-all-disappoints-everybody solution, maybe its time to build two effective solutions instead.

Re: Docker Compose best practices for dev and prod

#70

Post by fio_ini »

kh_hk wrote: post #53
fio_ini wrote: post #50

Earlier quoted context omitted.

your volume example is an anti-pattern in regard to something like the 12 factor app. you're conditionally changing and setting the volumes based on an environment your in? your code and service is gonna be busted. how would you even know if anything worked in prod if it's not reproducible in dev since # if dev then volume xyz else volume abc #

It's an anti-pattern just because you say so. This is a simplified example of a complex problem that needs arbitrary volume mounts. Using volume mounts to run software inside containers is a common dev pattern on scenarios where rebuilding takes too much time to quickly iterate. I am sorry, but unfortunately I cannot expand the issue further here to help you understand it, and I am happy too if you think I am over-en…

I honestly don't care if it's you misunderstanding or your teammates are building a bad framework/foundation to iterate upon. Doesn't matter to me either way bud. But you're going against the premise of docker and docker-compose which is write once and run anywhere because the services will be different every time you deploy since they'll have have variable path and file locations etc. - that's why it's an anti-pattern not because "i say so"
Post reply on HN