Why Docker Is Not Yet Succeeding Widely in Production • Hacker News Board

Re: Why Docker Is Not Yet Succeeding Widely in Production

#231

Post by AaronFriel » Wed, Jul 29, 2015, 1:06 AM UTC

I thought the obvious reason was storage. I don't see it mentioned here, but storage is a huge pain point. How do you store your critical data "with Docker" is a labyrinthine set of steps.

Docker's answer to storage so far has been "don't use Docker". That's their answer. Use volumes to map some other storage, but then you have to have some way of mapping storage to containers outside of Docker. Now you're really stuck.

Containers are awesome, but unless your product doesn't do work, you'll need to store data at some point. And that's when the magic stops.

Re: Why Docker Is Not Yet Succeeding Widely in Production

#232

Post by jesse_szwedko » Wed, Jul 29, 2015, 1:22 AM UTC

spoondan wrote: post #181

tracker1 wrote: post #172

Earlier quoted context omitted.

That's a bit of my point... If you're building relatively small independent services with Docker you can deploy service A with node 0.10 as it's tested environment and service B with iojs 2.4 on the same server, without them conflicting... when you need to update/enhance/upgrade service A you then can update the runtime. The same can be said for ruby, python and any number of other language environments where you hav…

> The same can be said for ruby, python and any number of other language environments where you have multiple services that were written at different times with differing base targets. I've seen plenty of instances where updating a host server to a new runtime breaks some service that also runs on a given server. This is a solved problem in Python and Ruby. In Python, use virtual environments. In Ruby, use RVM. You w…

It partially is, until you need native dependencies.

Re: Why Docker Is Not Yet Succeeding Widely in Production

#233

Post by kosma » Wed, Jul 29, 2015, 1:23 AM UTC

iamleppert wrote: post #227

I've said it before and I'll say it again: pre-mature infrastructure optimization is the root of all evil. Do me a favor and if you got a startup, stay clear of all this. Everyone wants to reinvent their own flavor of heroku and make your deployment and build pipeline god-awful complex. Their tool of choice? Docker. Before you know it you'll be swimming in containers upon containers. Containers will save us, they'll…

Until a few days ago I was a DevOps engineer at a medium-sized tech company. My primary responsibility was dockerizing their applications and infrastructure.

I quit the job.

The scenario played out just as you said: I ended up single-handedly and poorly re-engineering something that already existed (they did have a working Ansible setup) for no visible gain. "Swimming in containers upon containers" is exactly what happened; they kinda worked, but the farther we got, the more kludges piled on top of each other. In four months work we didn't even hit production - the most we got was a CI/QA service that was actually nothing more than a loose bunch of Python scripts. Between managing dev/test/prod differences, tracing missing logs, removing unused volumes, networking all that stuff together and trying to provide at least a decent level of security, I realized that I'm wasting everyone's time and money. Developers hated it because it filled their workflows with traps and obstacles. Admins hated it because of the lack of tooling. Business hated it because it caused unexplainable delays. The only thing we really accomplished was some compliance with the The Twelve-Factor App - something that could've been done in a week. Hardly a victory.

My advice? Forget about Docker unless your primary business is building hosting systems. It will take years before Docker gets mature enough for production, and not without a ton of tooling on top of it and some major architectural changes. Until then, go back to the old UNIX ways of doing things... it worked perfectly since the Epoch and it will continue to work long after the 32-bit time_t rolls over. You'll be fine.

Re: Why Docker Is Not Yet Succeeding Widely in Production

#234

Post by jesse_szwedko » Wed, Jul 29, 2015, 1:33 AM UTC

mariusmg wrote: post #42

Maybe because Docker isn't really needed ? I mean if your app needs the entire fucking OS to provide isolation from other apps, then you are clearly doing it wrong.

I think it's more that other people developing other apps don't want to have to worry about your app impacting them when colocated on the same server.

Re: Why Docker Is Not Yet Succeeding Widely in Production

#235

Post by mixmastamyk » Wed, Jul 29, 2015, 1:55 AM UTC

scurvy wrote: post #6

I still haven't been sold on Docker. Why would an otherwise competent company that runs things just fine, ditch it all and adopt Docker? Just because it's the shiny new thing? What do we actually gain here in production?

http://martinfowler.com/bliki/SnowflakeServer.html

Re: Why Docker Is Not Yet Succeeding Widely in Production

#236

Post by mixmastamyk » Wed, Jul 29, 2015, 2:00 AM UTC

emergentcypher wrote: post #203

why-el wrote: post #160

Earlier quoted context omitted.

Is your application just a jar? In the yes case your conclusion is correct. Throw in a database, a cache server, couple of versioned libraries your jar file needs, and more developers, and suddenly a reproducible image with all this packaged will make a lot of sense.

Our build process already includes the application's dependencies inside the jar, and packages it into an installable deb file that places the app, its config, and the init file in the right locations. I have a database and a cache server. They don't run on the same server as the application jar... they run on separate machines tuned to their purpose. Why would I want them packaged together? So my team doesn't have t…

How fast can you replace one of those servers when they go down? More info:

http://martinfowler.com/bliki/SnowflakeServer.html

Re: Why Docker Is Not Yet Succeeding Widely in Production

#237

Post by mixmastamyk » Wed, Jul 29, 2015, 2:06 AM UTC

DyslexicAtheist wrote: post #96

docker security issues[0] should really be listed as #1. The overhead/complexity of getting it secure (using SELinux) outweigh its benefits in production. [0] http://blog.valbonne-consulting.com/2015/04/14/as-a-goat-im-...

I don't think its a good choice if security is a main concern. I wouldn't put multi-tenant apps on it either.

Re: Why Docker Is Not Yet Succeeding Widely in Production

#238

Post by Nyetan » Wed, Jul 29, 2015, 2:31 AM UTC

kosma wrote: post #233

iamleppert wrote: post #227

I've said it before and I'll say it again: pre-mature infrastructure optimization is the root of all evil. Do me a favor and if you got a startup, stay clear of all this. Everyone wants to reinvent their own flavor of heroku and make your deployment and build pipeline god-awful complex. Their tool of choice? Docker. Before you know it you'll be swimming in containers upon containers. Containers will save us, they'll…

Until a few days ago I was a DevOps engineer at a medium-sized tech company. My primary responsibility was dockerizing their applications and infrastructure. I quit the job. The scenario played out just as you said: I ended up single-handedly and poorly re-engineering something that already existed (they did have a working Ansible setup) for no visible gain. "Swimming in containers upon containers" is exactly what ha…

Bother you for a little advice? I'm working at a mid-sized tech company and am evaluating Docker for CI, testing and limited, internal deployment usages.

The services in question are built with a hodge-podge of shell scripts and build tools, so getting them all to compile locally is a challenge, let alone deploying them. My hope was that containerizing the builds would isolate any configuration problems, and that containerizing the deployed services would cut down on outages by permitting trivial rollbacks (say, by snapshotting all the service containers before each deploy and merely restoring them should a deployment fail). Of course, all of the above could be fixed by traditional means (e.g. rewriting the build system with a single, standard tool; streamlining the deployment process, etc.), but it seemed like Docker could solve 80% of the problems while easing the implementation of the proper solutions down the line.

Considering the above, do you still think Docker's a poor fit for business that aren't building hosting systems? Oh, and any nuggets of wisdom you could throw to a newcomer to the industry? :)

Re: Why Docker Is Not Yet Succeeding Widely in Production

#239

Post by jordanthoms » Wed, Jul 29, 2015, 2:33 AM UTC

iamleppert wrote: post #227

I've said it before and I'll say it again: pre-mature infrastructure optimization is the root of all evil. Do me a favor and if you got a startup, stay clear of all this. Everyone wants to reinvent their own flavor of heroku and make your deployment and build pipeline god-awful complex. Their tool of choice? Docker. Before you know it you'll be swimming in containers upon containers. Containers will save us, they'll…

I do agree with you, but we're on Heroku and that also has it's downsides - the biggest one we've found is that CPU performance on 1x and 2x dynos is very unpredictable - We've had to move to Performance dynos ($500/month each!) to get decent performance and prevent random timeouts.

Services like Cloud66 are interesting (they manage deployments onto your own EC2 or other cloud infrastructure), but the developer experience doesn't quite match Heroku yet.

Heroku really needs some more competition...

Re: Why Docker Is Not Yet Succeeding Widely in Production

#240

Post by garthk » Wed, Jul 29, 2015, 3:22 AM UTC

davexunit wrote: post #173

api wrote: post #136

Earlier quoted context omitted.

I have mixed feelings about Docker. I've found three major use cases so far: (1) Testing. (2) Build environments -- it's helpful to build distribution Linux binaries in older Linux versions like CentOS 6 so that they'll work on a wider range of production systems. (3) Installing and running "big ball of mud" applications that want to drag in forty libraries, three different databases, memcached, and require a custom…

>Docker might still be nice for perfect reproducibility Docker actually doesn't help reproducibility at all, because the underlying reproducibility problems present in the distro and build systems are used are still present. See GNU Guix, Nix, and Debian's Reproducible Builds project for efforts to make build truly reproducible. I had a good laugh when I read "the Rube Goldberg Machine development anti-pattern". This…

> It's like static linking an entire operating system for each application.

You say it like it's a problem, but that's the most concise description of Docker I've yet read. It rhymes with the way all the fed up oldies using Go like its static linking.