Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you use Docker in production?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 157 posts

Re: Ask HN: How do you use Docker in production?

#111

Post by seppalala »

indygreg2 wrote: post #93
seppalala wrote: post #59

Earlier quoted context omitted.

Yes, very much so. In fact, our goal is to have NixOS based containers. Right now, we're using Debian as the base image, and there's /no/ guarantee that the versions of software installed are consistent (since Docker caches based on the line in a Dockerfile, rather than what's actually installed). With Nix, we can have version guarantees in all of our Docker images--including the cached images.

I blogged about this the other day and would love to hear about your experience! http://gregoryszorc.com/blog/2014/10/13/deterministic-and-mi...

This is spot-on what I'd like to do, and I have the exact same concerns. Thanks for sharing!

Re: Ask HN: How do you use Docker in production?

#112

Post by zenlikethat »

preillyme wrote: post #106

Docker explicitly violates the principles of the Twelve-Factor App. Docker apps don’t rely on any external environment. In fact, Docker demands that you store all config values, dependencies, everything inside of the container itself. Apps communicate with the rest of the world via ports and via Docker itself. The trade-off is that apps become a little bit bulkier (though not significantly), but the benefit is apps b…

Very good points, and as the author here (http://blog.lusis.org/blog/2014/06/14/paas-for-realists/) points out:

> I should state up front that I disagree with quite a few bits of the 12 Factor model. It’s important to remember that, imho, the 12 Factor model was designed as a business strategy for Heroku. The steps follow exactly what makes an app work best on Heroku, not what is best for an application.

Re: Ask HN: How do you use Docker in production?

#113

Post by HorizonXP »

Docker, CoreOS, fleet, and etcd have completely changed how I build projects. It's made me much more productive.

I'm working on Strata, which is a building management & commissioning system for property owners of high-rise smart buildings. It's currently deployed in a single building in downtown Toronto, and it's pulling in data from thousands of devices, and presenting it in real-time via an API and a dashboard.

So in this building, I have a massive server. 2 CPUs, 10 cores each, 128GB of RAM, the works. It came with VMWare ESXi.

I have 10 instances of CoreOS running, each identical, but with multiple NICs provisioned for each so that they can communicate with the building subsystems.

I built every "app" in its own Docker container. That means PostgreSQL, Redis, RabbitMQ, my Django app, my websocket server, even Nginx, all run in their own containers. They advertise themselves into etcd, and any dependencies are pulled from etcd. That means that the Django app gets the addresses for the PostgreSQL and Redis servers from etcd, and connects that way. If these values change, each container restarts itself as needed.

I also have a number of workers to crawl the network and pull in data. Deployment is just a matter of running 'fleetctl start overlord@{1..9}.service', and it's deployed across every machine in my cluster.

With this setup, adding machines, or adding containers is straightforward and flexible.

Furthermore, for development, I run the same Docker containers locally via Vagrant, building and pushing as needed. And when I applied for YC, I spun up 3 CoreOS instances on DigitalOcean and ran the fleet files there.

As I said, I've been able to streamline development and make it super agile with Docker & CoreOS. Oh, and I'm the only one working on this. I figure if I can do it on my own, imagine what a team of engineers can do.

Very powerful stuff.

Re: Ask HN: How do you use Docker in production?

#114

Post by zenlikethat »

grownseed wrote: post #110

Not technically in production yet, but I use Docker for the following scenarios: - Build agents for TeamCity, this was one of the first scenarios and it's been amazingly helpful so far. - Building third-party binaries in a reproducible environment - Running bioinformatics pipelines in consistent environments (using the above tools) - Circumventing the painfully inept IT department to give people in my group easy acce…

> Building third-party binaries in a reproducible environment

Awesome! Not enough people talk about this use case IMO. For instance, docker itself and the docker-cli are both compiled inside of a docker container. This allows you to circumvent the "OK, install the right version of Go, all the libraries I need etc." song and dance if you're developing in a new place (or if new developers join in the contribution).

It's also a great way to circumvent the (all to familiar) problem of "OK which lib* do I need to get this to compile? It isn't documented" problem.

Re: Ask HN: How do you use Docker in production?

#115

Post by neurotech1 »

I've used Docker in semi-production environments.

Configuration abd dependency management is much improved, and more efficient than VMs.

YAML configuration is easy to hand edit.

Docker doesn't have to rebuild an entire image for a minor application code or conf change. Incremental cache speeds up the build process.

Scaleout with "worker" instances is quite easy to manage.

For full production Elastic Beanstalk is worth a look. I prefer to host on DigitalOcean VMs for dev staging.

Docker has a great local community in San Francisco.

Re: Ask HN: How do you use Docker in production?

#117

Post by ownagefool »

HorizonXP wrote: post #113

Docker, CoreOS, fleet, and etcd have completely changed how I build projects. It's made me much more productive. I'm working on Strata, which is a building management & commissioning system for property owners of high-rise smart buildings. It's currently deployed in a single building in downtown Toronto, and it's pulling in data from thousands of devices, and presenting it in real-time via an API and a dashboard. So…

Sounds like it works for you, but you should probably think about redundancy in your hardware there.

Re: Ask HN: How do you use Docker in production?

#118

Post by zenlikethat »

zwischenzug wrote: post #100
jjd1103 wrote: post #39

Earlier quoted context omitted.

Docker also enforces immutability. With a VM there's always the temptation to manually fix any issue that arises, and if you don't have some bulletproof way to document that then you'll have issues when you go to recreate the environment on a new machine. Docker kind of forces you to solve the original problem via the dockerfile, which is what will spawn images for any future installs anyway.

Only true if you don't go to the network: http://zwischenzugs.wordpress.com/2014/07/16/phoenix-deploym...

Very good call out. For most use cases this actually turns out to be OK, but to reduce the surface area of this being a potential issue you could:

- Vendor dependencies (works to replace stuff like `go get` but probably not for apt packages etc.)

- Create a base image which handles the stuff you need to reach out to the network for (`apt-get install openjdk-6-jre` etc.) and is infrequently updated. Then the Dockerfile for the final application is `FROM me/myjava` and just does a few things that don't use the network like `ADD . /code`.

- Use `docker commit` instead of Dockerfiles for those steps (pretty gross IMO)

- Use CM in your docker build to install a very specific version of a package if you need (I'm not 100% sure this exists but it seems probable). This isn't perfect but tightens things up if you're worried about upstream breaking apt packages etc.

One of the goals of a new image format for Docker (this is 2.0 stuff) is to make the layers content-addressable by ID. That way, you will have a reasonable assurance that two Docker images constructed with the same Dockerfile in two different places will have the same IDs if they result in the exact same layers, and you will be able to see the point of divergence otherwise.

Re: Ask HN: How do you use Docker in production?

#119

Post by MattyMc »

wvanbergen wrote: post #65
MattyMc wrote: post #34

Earlier quoted context omitted.

wvanbergen, forgive me for veering off topic. I'm planning on applying to Shopify (Toronto) as a software developer before the end of the weekend. Any advice you're willing to share?

MattyMc: sure. Primarily: be yourself, show what you are passionate about, and be willing to adopt change. When applying: cover letter > resumé, and try to stand out because we get many applications. Email me at willem at shopify dot com if you have any specific questions.

Thanks so much Willem!

Re: Ask HN: How do you use Docker in production?

#120

Post by positivejam »

grownseed wrote: post #110

Not technically in production yet, but I use Docker for the following scenarios: - Build agents for TeamCity, this was one of the first scenarios and it's been amazingly helpful so far. - Building third-party binaries in a reproducible environment - Running bioinformatics pipelines in consistent environments (using the above tools) - Circumventing the painfully inept IT department to give people in my group easy acce…

> - Build agents for TeamCity, this was one of the first scenarios and it's been amazingly helpful so far.

Can you expand on what you've done with them and how it's been helpful?

Post reply on HN