Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker in Production: An Update

thehftguy.com

Post reply on HN
Print view View original

71–80 of 97 posts

Re: Docker in Production: An Update

#71

Post by daenney »

Artemis2 wrote: post #6

> Google merely exposes a Docker interface, all the containers are run on internal google containerization technologies, that cannot possibly suffer from all the Docker implementation flaws. Google running their own containerization tech with a Docker interface seems a bit far-fetched given the level of integration of Kubernetes with Docker. That's totally possible though, I'd like to read more about it.

From one of the GKE folks: https://news.ycombinator.com/item?id=13715358

It's not true.

Re: Docker in Production: An Update

#72

Post by dekhn »

crb wrote: post #60
dekhn wrote: post #35

The claim that GKE doesn't use Docker internally is false. You can ssh to the master and see all the docker containers.

You can't ssh to the master with GKE, but you can check the nodes (as mentioned at https://news.ycombinator.com/item?id=13715358. )

Sorry, yes. You used to be able to ssh to the master.

Re: Docker in Production: An Update

#73

Post by cjhanks »

mwpmaybe wrote: post #36
cjhanks wrote: post #32

Earlier quoted context omitted.

I have tried to use Docker on very busy processing systems with short to medium running tasks. My efforts were on Ubuntu 16.04... because any non-LTS is almost a non-starter. There were two major obstacles; the file system driver and the Docker daemon. I ended up settling on 1.11.{i forget}, because virtually every other version was unusable. As sad as it may sounds, I was tracking two metrics; probability the contai…

Were you using Device Mapper purposefully, or just because you couldn't get a more typical storage driver to work? The general setup instructions recommend either 1. installing linux-image-extra and using AUFS (for kernel versions = 4). Admittedly, the latter strategy is a bit frustrating and counterintuitive on 16.04. When you install Docker it will start automatically and hang because it can't find the AUFS driver…

I mean to say, device mapper was the only solution which worked in a production setting for me.

I no longer work with that company and so I can't tell how those improvements would change the reliability. Presently I am using docker with btrfs on a low-throughput workload and it seems to work just fine.

Tangentially related - I have been using OverlayFS without Docker for about a year now. It's pretty great. My read-layers can be on an NFS drive and the penalty for writes is relatively small. It basically gives me 90% of what I wanted from Docker in the first place.

Re: Docker in Production: An Update

#74

Post by StreamBright »

_Codemonkeyism wrote: post #65

While other people just use Docker to serve hundreds of thousands of users (mid size startup) and make money.

Yeah like running a CI job for customers. Absolutely great with Docker, nobody cares about long term reliability. Container crashes? Now worries we spin up 10 more and re-start the CI job. Perfect match for Docker.

Running financial infra is quite the opposite. Long term reliable processes, not much space left for failures and mishaps, especially not in the lowest level of your infrastructure. What is the benefit for using Docker for this sort of services? Almost zero.

Re: Docker in Production: An Update

#76

Post by danielbln »

StreamBright wrote: post #53
danielbln wrote: post #5

Earlier quoted context omitted.

We've been running docker in production since late 2013 (data science SaaS). We had some growing pains in the beginning, but since then it has been smooth sailing and a tremendous help in getting our services deployed reliably. However, we have always been very conservative when it comes to upgrading docker and have our own custom glue in place. Still, the statement that "no-one on the planet using Docker seriously A…

Size of your infra would he helpful.

Pretty benign, 15 beastly bare metal machines in coloction and a bunch of VMs in AWS. A few database servers in both.

Re: Docker in Production: An Update

#77

Post by user5994461 »

crb wrote: post #56
wmf wrote: post #50

Earlier quoted context omitted.

So what magic combination of distro + FS do you use to make Docker stable?

The default distro is Container Optimised OS: https://cloud.google.com/container-optimized-os/docs/ . It's derived from Chromium OS, which means we can take advantage of the team who build images for the many devices which use it, and the security response infrastructure around it. With 'docker info' on a node you can see we use OverlayFS, which seems a popular choice in the community also: http://burkelibbey.s3.amaz…

So, customized Google OS, but using the official Docker package? Oh wait. How could there be an official docker package for an OS noone known existed?

I assume customized kernel as well? and where does the overlay drivers come from? How much custom back-ports and custom development?

The article is not on point to say GKE replaced docker entirely then... but you are not point to deny and pretend that you are running Docker on anything remotely common.

Re: Docker in Production: An Update

#78

Post by [deleted] »

crb wrote: post #9

It's great that hftguy thinks Google Container Engine is stable (I work on it) but I'm sorry to say it's very easy to prove that is, in fact, running Docker on the nodes. You can just SSH into one and see for yourself. Kubernetes was built from the ground up to orchestrate Docker. CoreOS did a lot of work to make it possible to trade rkt in for Docker's engine, and the cri (Container Runtime Interface) is now general…

[deleted]

Re: Docker in Production: An Update

#79

Post by ownagefool »

StreamBright wrote: post #54
fnbr wrote: post #8

Earlier quoted context omitted.

We're technically using it in production for my site, www.bugdedupe.com, but we're still in beta, so we haven't experienced heavy usage. We've been using Kubernetes to deploy Docker containers on Google Container Engine, and while there have been a few issues due to Docker/Kubernetes (namely, getting the containers to expose localhost to each other, and to expose themselves to the world), the issues that we've had so…

Size of your infra would he helpful to know.

I don't know the above poster but we run over 4500 docker containers on AWS distributed using kubernetes. Docker has indeed caused us a problem or two in the past, but actually it's largely smooth sailing and most of our problems are with the app layers and both docker and kubernetes have largely made our lives much easier.

A quick skim would suggests that sits on about 8TB of Ram and 1000vCPUs spread over 100 machines. It's not the biggest stack in the world, but it's doing a good job as our little internal PaaS.

We have some non-aws versions too, but I don't have metrics on those right now.

Re: Docker in Production: An Update

#80

Post by user5994461 »

StreamBright wrote: post #52
bitwalker wrote: post #31

> CoreOS is an operating that can only run Docker and is exclusively intended to run Docker. This shows an astonishing level of ignorance for someone who claims to have done their research. > First, the main benefit of Docker is to unify dev and production. Having a separate OS in production only for containers totally ruins this point. What? This makes no sense. Your images will be the same between dev and prod, eve…

>> 120+ containers on 12 hosts it is virtually irrelevant what you use to run and operate this sort of cluster. it gets tricky when you pass 100 nodes and even trickier when you get to 1000+ nodes in a non-linear fashion. Docker certainly has issues that are pretty severe when you are in the financial space and not a big deal if you are running a popular blog like medium.com for example.

12 nodes running 12 containers (1 per node) each bringing in 12$ a second and you're a multi billion dollars company.

Let's multiply that a few times for dev, test and support systems. Still, no need to have hundreds of nodes.

Post reply on HN