Vagrant Feature Preview: Docker-Based Development Environments

This is the best piece of news to start my Wednesday morning that I could've asked for. I've been slowly converting the entire team in the office over to Vagrant, and am using it for everything. I recently started playing with Docker and wanted to explore deployment of our web apps using it, and now I'll be able to slot it in to my existing workflow! Vagrant is amazing :)

This is a really great step forward - thanks Mitchell!

I've recently spent a couple of weeks doing a deep dive into Docker, so I'll share some insights from what I've learned.

First, it's important to understand that Docker is an advanced optimization. Yes, it's extremely cool, but it is not a replacement for learning basic systems first. That might change someday, but currently, in order to use Docker in a production environment, you need to be a pro system administrator.

A common misconception I see is this: "I can learn Docker and then I can run my own systems with out having to learn the other stuff!" Again, that may be the case sometime in the future, but it will be months or years until that's a reality.

So what do you need to know before using Docker in production? Well, basic systems stuff. How to manage linux. How to manage networking, logs, monitoring, deployment, backups, security, etc.

If you truly want to bypass learning the basics, then use Heroku or another similar service that handles much of that for you. Docker is not the answer.

If you already have a good grasp on systems administration, then your current systems should have:

    - secured least-privilege access (key based logins, firewalls, fail2ban, etc)
    - restorable secure off-site database backups
    - automated system setup (using Ansible, Puppet, etc)
    - automated deploys
    - automated provisioning
    - monitoring of all critical services
    - and more (I'm writing this on the fly...)

If you have critical holes in your infrastructure, you have no business looking at Docker (or any other new hot cool tools). It'd be like parking a Ferrari on the edge of an unstable cliff.

Docker is amazing - but it needs a firm foundation to be on.

Whenever I make this point, there are always a few engineers that are very very sad and their lips quiver and their eyes fill with tears because I'm talking about taking away their toys. This advice isn't for them, if you're an engineer that just wants to play with things, then please go ahead.

However, if you are running a business with mission-critical systems, then please please please get your own systems in order before you start trying to park Ferraris on them.

So, if you have your systems in order, then how should you approach Docker? Well, first decide if the added complexity is worth the benefits of Docker. You are adding another layer to your systems and that adds complexity. Sure, Docker takes care of some of the complexity by packaging some of it beautifully away, but you still have to manage it and there's a cost to that.

You can accomplish many of the benefits of Docker without the added complexity by using standardized systems, ansible, version pinning, packaged deploys, etc. Those can be simpler and might be a better option for your business.

If the benefits of Docker outrank the costs and make more sense than the simpler cheaper alternatives, then embrace it! (remember, I'm talking about Docker in production - for development environments, it's a simpler scenario)

So, now that you've chosen Docker, what's the simplest way to use it in production?

Well, first, it's important to understand that it is far simpler to manage Docker if you view it as role-based virtual machine rather than as deployable single-purpose processes. For example, build an 'app' container that is very similar to an 'app' VM you would create along with the init, cron, ssh, etc processes within it. Don't try to capture every process in its own container with a separate container for ssh, cron, app, web server, etc.

There are great theoretical arguments for having a process per container, but in practice, it's a bit of a nightmare to actually manage. Perhaps at extremely large scales that approach makes more sense, but for most systems, you'll want role-based containers (app, db, redis, etc).

If you're still not convinced on that point, read this on microservices which points out many of the management problems: http://highscalability.com/blog/2014/4/8/microservices-not-a...

You probably already have your servers set up by role, so this should be a pretty straight-forward transition. Particularly since you already have each system scripted in Ansible (or similar) right?

To run Docker in a safe robust way for a typical multi-host production environment requires very careful management of many variables:

    - secured private image repo (index)
    - orchestrating container deploys with zero downtime
    - orchestrating container deploy roll-backs
    - networking between containers on multiple hosts
    - managing container logs
    - managing container data (db, etc)
    - creating images that properly handle init, logs, etc
    - much much more...

This is not impossible and can all be done and several large companies are already using Docker in production, but it's definitely non-trivial. This will change as the ecosystem around Docker matures (Flynn, Docker container hosting, etc), but currently if you're going to attempt using Docker seriously in production, you need to be pretty skilled at systems management and orchestration.

There's a misconception that using Docker in production is nearly as simple as the trivial examples shown for sample development environments. In real-life, it's pretty complex to get it right. For a sense of what I mean, see these articles that get the closest to production reality that I've found so far, but still miss many critical elements you'd need:

http://code.tutsplus.com/tutorials/easily-deploy-redis-backe...

http://www.powpark.com/blog/programming/2014/01/29/integrati...

http://blog.buddycloud.com/post/80771409167/using-docker-wit...

https://journal.paul.querna.org/articles/2013/10/15/docker-u...

(If you know of better ones, please share!)

To recap, if you want to use Docker in production:

    1. Learn systems administration
    2. Ensure your current production systems are solid
    3. Determine whether Docker's benefits justifies the cost
    4. Use role-based containers

Shameless plug: I'll be covering how to build and audit your own systems in more depth over the next couple months (as well as more Docker stuff in the future) on my blog. If you'd like to be notified of updates, sign up on my mailing list: https://devopsu.com/newsletters/devopsu.html

Although I've heard of Fig, I just took a closer look for the first time. From the docs, the Docker provider is a super-set of it. Meaning: Vagrant + Docker can do everything Fig can do and then some. I may have missed some detail, but at the very least all of the major pieces are there.

"just a list of commands to run on the machine."

A provisioning script is essentially just a list of idempotent commands to run on the machine. But given the way that docker works, idempotence is not required -- if you change a command, Docker rolls back to a known state and runs the command.

A provisioning script might be slightly "higher-level" than a Dockerfile or shell script, but I find the difference is minimal, that the number of lines of code required are similar. Many provisioning tools provide libraries of pre-built recipes you can utilize; Docker provides a repository of pre-built images.

> Docker runs fine with multiple processes in a container.

I think that, while most people realize this, it's important to highlight this fact again. Generally, the examples you see are "look, we can run this would-be-backgrounded-elsewhere daemon in the foreground in its own container!", when IMO this sets the bar a bit low for how people initially approach Docker. Containers have simplified my life significantly, and (again, IMO) their real power becomes evident when you treat them as logical units vs individual components.