Docker in Production: An Update • Hacker News Board

Re: Docker in Production: An Update

#31

Post by bitwalker » Thu, Feb 23, 2017, 5:36 PM UTC

> CoreOS is an operating that can only run Docker and is exclusively intended to run Docker.

This shows an astonishing level of ignorance for someone who claims to have done their research.

> First, the main benefit of Docker is to unify dev and production. Having a separate OS in production only for containers totally ruins this point.

What? This makes no sense. Your images will be the same between dev and prod, even if the host running the containers is different - which is really the whole point - if you build and run an image in dev, it should run identically in prod.

> If you like playing with fire, it looks like that’s the OS of choice.

We spent the last year+ running containers on Centos7 with no problems from the OS. Whatever issues we did encounter were either transient bugs with Docker or our own configuration. Perhaps we got super lucky, but we were running 120+ containers on 12 hosts, so I would've expected at least some evidence of significant problems within that timeframe if it were really such a risky setup.

> It’s not possible to build a stable product on a broken core, yet both Pivotal and RedHat are trying.

We've been running OpenShift Origin since March of last year, it's been very stable during that time - the few issues we did encounter were due to our own mistakes, and were usually fixed just by changing some configuration and restarting the host.

While there are undoubtedly problems with Docker, and likely many of the issues you brought up are very real, there are many teams like mine that use it successfully, and painlessly. Docker isn't the tire fire you want to make it out to be.

Re: Docker in Production: An Update

#32

Post by cjhanks » Thu, Feb 23, 2017, 5:45 PM UTC

mwpmaybe wrote: post #10

I am neither a Docker expert nor evangelist, and I have my own gripes and frustrations with it, but this article is full of misinformation and FUD. To wit: > CoreOS is an operating [system] that can only run Docker and is exclusively intended to run Docker. No, it isn't. (Maybe if you substitute "Docker" with "containers.") > First, the main benefit of Docker is to unify dev and production. Having a separate OS in pr…

I have tried to use Docker on very busy processing systems with short to medium running tasks. My efforts were on Ubuntu 16.04... because any non-LTS is almost a non-starter.

There were two major obstacles; the file system driver and the Docker daemon. I ended up settling on 1.11.{i forget}, because virtually every other version was unusable. As sad as it may sounds, I was tracking two metrics; probability the container will start and time until the docker daemon reaches a state of irrecoverable dead-lock.

Personally I found containers (on a highly contentious system) launched at ~98% success rate with a time-to-dead-lock somewhere around 6500 container start/stop (it was a pretty fat tailed distribution). For a busy system... those numbers equate to an administrative headache.

And on Ubuntu/Debian the defaults (the thing the majority of non-specialists are probably using) were far worse. The only filesystem driver which seemed to work at all was devicemapper direct-lvm.

-

If you have developers on your team who can spend their time figuring out the one magic incantation that makes Docker work most of the time, you're fine. Everybody else should follow his advice for using service providers.

Re: Docker in Production: An Update

#33

Post by dastbe » Thu, Feb 23, 2017, 5:49 PM UTC

mdekkers wrote: post #19

Every three months or so I tell myself "right,get your finger out, and do something production quality with containers" and come to the same conclusion this guy did.

production quality as in "in production" or as in "something that i will actually use in my day to day"?

Re: Docker in Production: An Update

#34

Post by shykes » Thu, Feb 23, 2017, 5:55 PM UTC

holydude wrote: post #17

I still think that Solaris zones are / were a superior technology. Sadly it died with the demise of Solaris as an operating system. Still in use in some "must have enterprise" types of companies. I would not run anything serious on docker (definitely not yet) if I would not be heavily invested in understanding and contributing to the technology (docker codebase + related). Docker is a nice to have but absolutely abst…

> I am wondering what people like Brendand Gregg think about docker.

He will be speaking at Dockercon in April, so I suppose we'll find out soon enough.

Re: Docker in Production: An Update

#35

Post by dekhn » Thu, Feb 23, 2017, 5:57 PM UTC

The claim that GKE doesn't use Docker internally is false. You can ssh to the master and see all the docker containers.

Re: Docker in Production: An Update

#36

Post by mwpmaybe » Thu, Feb 23, 2017, 5:58 PM UTC

cjhanks wrote: post #32

mwpmaybe wrote: post #10

I am neither a Docker expert nor evangelist, and I have my own gripes and frustrations with it, but this article is full of misinformation and FUD. To wit: > CoreOS is an operating [system] that can only run Docker and is exclusively intended to run Docker. No, it isn't. (Maybe if you substitute "Docker" with "containers.") > First, the main benefit of Docker is to unify dev and production. Having a separate OS in pr…

I have tried to use Docker on very busy processing systems with short to medium running tasks. My efforts were on Ubuntu 16.04... because any non-LTS is almost a non-starter. There were two major obstacles; the file system driver and the Docker daemon. I ended up settling on 1.11.{i forget}, because virtually every other version was unusable. As sad as it may sounds, I was tracking two metrics; probability the contai…

Were you using Device Mapper purposefully, or just because you couldn't get a more typical storage driver to work? The general setup instructions recommend either 1. installing linux-image-extra and using AUFS (for kernel versions = 4).

Admittedly, the latter strategy is a bit frustrating and counterintuitive on 16.04. When you install Docker it will start automatically and hang because it can't find the AUFS driver and Device Mapper isn't set up. The solution is to either 1. modify policy-rc.d to prevent services from automatically starting or 2. set up Device Mapper (install dmsetup and run `dmsetup mknodes`) before installing Docker and changing the storage driver. Unfortunately, these workarounds are not particularly well-documented.

I would be interested to see how Docker 1.12 or 1.13 with a modern kernel and OverlayFS would be able to handle your described workload.

Re: Docker in Production: An Update

#37

Post by random3 » Thu, Feb 23, 2017, 6:00 PM UTC

I think Docker was and still is a great way to popularize the concept of containers along with the smarter tooling around it, like resource schedulers. By the time we adopted Docker (because it was popular and realized it's a good vehicle to push some concepts across organizations) we knew we're going to use it as a package system in a distributed scheduling environment (based on Mesos, Zookeeper, HDFS) and that it may get replaced later on. However, it took longer for Docker Inc. and the community to depart from the original ideas (initially very developer task centric, including trying to figure out how to store data inside containers) and figure how to enable scheduling, service discovery, health checking etc. I think Kubernetes did a good job popularizing better practices.

IMO two of the few systems that were sound were Mesos and Kubernetes (and their roots are somehow interleaved) and neither puts the Docker as the central piece, nor the container, which is just a building block to achieve the actual goals of running distributed, highly available jobs and services efficiently in a shared environment.

Re: Docker in Production: An Update

#38

Post by Veratyr » Thu, Feb 23, 2017, 6:15 PM UTC

bitwalker wrote: post #31

> CoreOS is an operating that can only run Docker and is exclusively intended to run Docker. This shows an astonishing level of ignorance for someone who claims to have done their research. > First, the main benefit of Docker is to unify dev and production. Having a separate OS in production only for containers totally ruins this point. What? This makes no sense. Your images will be the same between dev and prod, eve…

> What? This makes no sense. Your images will be the same between dev and prod, even if the host running the containers is different - which is really the whole point - if you build and run an image in dev, it should run identically in prod.

I think the author is saying that while this is the point, it's not the reality. As it is right now, things can run absolutely fine in dev and then due to prod running a different operating system, things break.

Re: Docker in Production: An Update

#39

Post by Draiken » Thu, Feb 23, 2017, 6:15 PM UTC

crb wrote: post #9

It's great that hftguy thinks Google Container Engine is stable (I work on it) but I'm sorry to say it's very easy to prove that is, in fact, running Docker on the nodes. You can just SSH into one and see for yourself. Kubernetes was built from the ground up to orchestrate Docker. CoreOS did a lot of work to make it possible to trade rkt in for Docker's engine, and the cri (Container Runtime Interface) is now general…

We very recently moved from some bare-metal pet machines into Google GKE and couldn't be happier.

Honestly the hardest thing is keeping up with how fast Kubernetes evolves and gets better and better. The same goes for all Google services (pubsub, bigquery, etc)

We started the migration on Kubernetes 1.1 and are now live on 1.5.1

Even using it for things we probably shouldn't (old stateful applications) without a single problem. At least no Docker related problems.

I don't know, this article seems to be very presumptuous. A lot of bold claims, little backing and, as you state, some pretty false claims.

TBH I don't know why it's among the top of HN

Re: Docker in Production: An Update

#40

Post by user5994461 » Thu, Feb 23, 2017, 6:40 PM UTC

mwpmaybe wrote: post #36

cjhanks wrote: post #32

Earlier quoted context omitted.

I have tried to use Docker on very busy processing systems with short to medium running tasks. My efforts were on Ubuntu 16.04... because any non-LTS is almost a non-starter. There were two major obstacles; the file system driver and the Docker daemon. I ended up settling on 1.11.{i forget}, because virtually every other version was unusable. As sad as it may sounds, I was tracking two metrics; probability the contai…

Were you using Device Mapper purposefully, or just because you couldn't get a more typical storage driver to work? The general setup instructions recommend either 1. installing linux-image-extra and using AUFS (for kernel versions = 4). Admittedly, the latter strategy is a bit frustrating and counterintuitive on 16.04. When you install Docker it will start automatically and hang because it can't find the AUFS driver…

So you are saying that all default settings are unusable in production, to the point docker might not start at all, and the only cure is series of 5 obscure advanced system setup/configuration that are almost impossible to figure out by oneself and not documented, yet they should be totally obvious to anyone using Docker, right?

FYI: It's because of this sort of bullshit that there are articles called "Docker in Production: An history of failure".