Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons learned from using Docker Swarm mode in production

blog.bugsnag.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Lessons learned from using Docker Swarm mode in production

#11

Post by leetrout »

gkze wrote: post #8
andrenth wrote: post #6

Earlier quoted context omitted.

The one feature that to me seems to be essential but appears to be missing from all these container orchestrators is the ability to tie a remote volume (Ceph/Gluster/Lustre/etc) to a container so that if a container is scheduled to run on a certain node, the volume will automatically be mounted on the same node. It seems from the mailing list that at least Nomad will have that at some point, but I have not seen much…

we tried using EFS for shared storage but quickly depleted our I/O bursting credits and our throughput dropped to a grinding halt, because our app's worklaod is both disk write and read intensive. No solutions yet

Unrelated, just some color commentary, I put my first request that took over 24 hours to be approved... 150k+ provisioned iops. Seems they don't like doing that.

Re: Lessons learned from using Docker Swarm mode in production

#12

Post by leetrout »

gkze wrote: post #4
leetrout wrote: post #2

I am a broken record lately on here... Nomad is really awesome for avoiding configuration hell and having to manage multiple services for container orchestration. It's a single binary and very very easy to setup and run. I actually prefer it to Swarm but YMMV. It uses Consul under the hood and has so far been bulletproof. (They all have their drawbacks / idiosyncrasies). https://www.nomadproject.io/

just curious - does it provide rolling updates to the jobs that you're running currently?

Nomad can do it but I've not used that feature.

Re: Lessons learned from using Docker Swarm mode in production

#13

Post by gkze »

leetrout wrote: post #2

I am a broken record lately on here... Nomad is really awesome for avoiding configuration hell and having to manage multiple services for container orchestration. It's a single binary and very very easy to setup and run. I actually prefer it to Swarm but YMMV. It uses Consul under the hood and has so far been bulletproof. (They all have their drawbacks / idiosyncrasies). https://www.nomadproject.io/

The point I'm trying to make is that these clustering capabilities are directly inside docker, so you don't have to run anything else but Docker. That is a huge win for us

Re: Lessons learned from using Docker Swarm mode in production

#14

Post by andrenth »

tristanz wrote: post #9
andrenth wrote: post #6

Earlier quoted context omitted.

The one feature that to me seems to be essential but appears to be missing from all these container orchestrators is the ability to tie a remote volume (Ceph/Gluster/Lustre/etc) to a container so that if a container is scheduled to run on a certain node, the volume will automatically be mounted on the same node. It seems from the mailing list that at least Nomad will have that at some point, but I have not seen much…

Kubernetes supports this via PersistentVolumes. Supported types include: GCEPersistentDisk AWSElasticBlockStore AzureFile FC (Fibre Channel) NFS iSCSI RBD (Ceph Block Device) CephFS Cinder (OpenStack block storage) Glusterfs VsphereVolume HostPath (single node testing only – local storage is not supported in any way and WILL NOT WORK in a multi-node cluster) http://kubernetes.io/docs/user-guide/persistent-volumes/

Interesting, thanks! To be honest I haven't looked too much into Kubernetes so far because of the emphasis in cloud deployment, while my interest is in setting it up in bare metal.

This feature might make me give it a try though!

Re: Lessons learned from using Docker Swarm mode in production

#15

Post by gkze »

bandrami wrote: post #7

Nothing whose version ends in "-rc4" is used in "production". You're using it in a very hot beta test.

We ran RC4 in production and we're running 1.12.1GA in production right now as well. We have been making money while running this and serving live customer traffic so we consider it production :)

Re: Lessons learned from using Docker Swarm mode in production

#17

Post by doublerebel »

rb808 wrote: post #16

While we're on the topic can anyone recommend a system for rolling out (Java) applications across server farms that doesn't use containers? We have a bunch of shell scripts that are pretty horrible. We could containerize, but we dont need that right now.

Distelli. It'll be a direct translation of your shell scripts to their format. I've had a great experience with them.

Re: Lessons learned from using Docker Swarm mode in production

#18

Post by tdurden »

rb808 wrote: post #16

While we're on the topic can anyone recommend a system for rolling out (Java) applications across server farms that doesn't use containers? We have a bunch of shell scripts that are pretty horrible. We could containerize, but we dont need that right now.

Ansible. It isn't perfect, but is far better than shell scripts for application deployment.

Re: Lessons learned from using Docker Swarm mode in production

#20

Post by colemickens »

andrenth wrote: post #14
tristanz wrote: post #9

Earlier quoted context omitted.

Kubernetes supports this via PersistentVolumes. Supported types include: GCEPersistentDisk AWSElasticBlockStore AzureFile FC (Fibre Channel) NFS iSCSI RBD (Ceph Block Device) CephFS Cinder (OpenStack block storage) Glusterfs VsphereVolume HostPath (single node testing only – local storage is not supported in any way and WILL NOT WORK in a multi-node cluster) http://kubernetes.io/docs/user-guide/persistent-volumes/

Interesting, thanks! To be honest I haven't looked too much into Kubernetes so far because of the emphasis in cloud deployment, while my interest is in setting it up in bare metal. This feature might make me give it a try though!

There are plenty of people running Kubernetes on bare-metal. There are a number of resources out there for PXE-booting a Kubernetes cluster on bare-metal.

Might be time to take a closer look ;).

Post reply on HN