Container technologies at Coinbase: Why Kubernetes is not part of our stack

Post by anang » Tue, Jun 09, 2020, 8:53 AM UTC

wwright wrote: post #101

zaptheimpaler wrote: post #75

Earlier quoted context omitted.

Plain old linux is the alternative, which is also "learn once, use everywhere" whether its AWS EC2 or GCP Instances or nearly any machine under the sun. I don't see how k8s avoids the need to learn about cloud vendor specific tech. e.g searching "aws RDS k8s" gives me a beta github packages and a bunch of blog posts on how to configure it right. It doesn't sound like much less work than learning how to use RDS withou…

“Plain old Linux” really isn’t an alternative to K8S though. You would need a load balancer, service discovery, a distributed scheduler, a configuration management system (K8S is a very strong alternative to building things around Ansible IMO). You can do all of those things without K8S, of course, but not with “plain old Linux” (what would that be anyway? GNU coreutils and the kernel? Vanilla Debian stable?)

Post by p_l » Tue, Jun 09, 2020, 8:57 AM UTC

dijit wrote: post #233

ec109685 wrote: post #184

Earlier quoted context omitted.

One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own lar…

Isn't that the benefit of cloud? Maybe I'm spoiled by GCP and it's not the same in other providers; but I can have a brand new debian VM configured and operational in less than 10s (less the time that I run my own startup script). Debian machines spawn with incredible speed; not much slower (if at all slower) than a new container.

Post by hardwaresofton » Tue, Jun 09, 2020, 9:02 AM UTC

jen20 wrote: post #163

flowerlad wrote: post #100

Earlier quoted context omitted.

> Plain old linux is the alternative How do you run an application on a cluster of plain old linux machines? How do you do load balancing? How do you scale up and down? How do you update your app without downtime? How do you roll back easily if something goes wrong? How do you ensure all your servers are running the same version of dependencies? How do you update those dependencies? How do you replicate your environm…

These are commonly raised concerns, all of which have answers much simpler than "install this giant distributed system". I'll go ahead and answer them since I take the questions to be in good faith... > How do you run an application on a cluster of plain old linux machines? Build a package, install it in an image, run that image in an autoscaling group (or whatever equivalent your cloud of choice offers). > How do yo…

Post by lmm » Tue, Jun 09, 2020, 9:03 AM UTC

hwbehrens wrote: post #76

joana035 wrote: post #62

Earlier quoted context omitted.

K8s is a nice api on top of gnu/linux. Want a iptables rule? Write a yaml (network policy). Want a storage for you app? Write a yaml (persistent volume) etc etc. For people who already know Linux, kubernetes comes naturally because it is pretty obvious. But indeed, by experience, many companies can go to "unicorn scale" with two or three boxes.

Couldn't you get nearly the same behavior using some basic Ansible playbooks? My impression was that the killer feature of k8s was scaling, automatic failover, etc., although to be fair it's been several years since I last looked into it.

Post by p_l » Tue, Jun 09, 2020, 9:09 AM UTC

namelosw wrote: post #230

Kubernetes is great but most projects don't really need it. Better go with Heroku equivalents than feeding a whole team, before you really need a whole team to feed Kubernetes.