Container technologies at Coinbase: Why Kubernetes is not part of our stack

> We would need to build/staff a full-time Compute team

This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase.

It was a nightmare. If you want k8s, you really do need people that know how to maintain it on a more or less full time schedule. Kubernetes is really not the magic bullet it's billed to be.

> Managed Kubernetes (EKS on AWS, GKE on Google) is very much in its infancy and doesn’t solve most of the challenges with owning/operating Kubernetes (if anything it makes them more difficult at this time)

Oh man this hits home. EKS is an absolute shitshow and their upgrade schedule is (a) not reliable, and (b) incredibly opaque. Every time we did a k8s version bump, we'd stay up the entire night to make sure nothing broke. We've since migrated to cloud functions (on GCP; but AWS lambdas could also work) and it's just been a breeze.

I also want to add that "auto-scaling" is one of the main reasons people are attracted to Kubernetes.. but in a real life scenario running like 2000 pods with an internal DNS, and a few redis clusters, and Elastic Search, and yadda yadda... it's a complete pain in the butt to actually set up auto-scaling. Oh, also, the implementation of Kubernetes cron jobs is also complete garbage (spawning a new pod every job is insanely wasteful).

- Hypertext blog post with some pictures that takes six seconds to load.

- Exchange that crashes during any time of high volume.

"Here is why, in our superior engineering judgment, you should do things the way we do in order to preserve uptime."

(To answer you question, probably because it uses the Medium code.)

Can fully relate to the author. We have been struggling with effective k8s at our company for 2 years now. Too much to learn to get your first service in production. You will end up writing wrappers after wrappers that make you think “we can’t be the first guys to solve this” and after googling, you find that every issue you find is described as “just” one of the downsides of using k8s and here’s how we overcame it.

I wish someone can make a deployment orchestrator for your private DC that is as simple to use as Heroku is. Edit: fixed typo.

> This isn't really about the article, but why does it take 6 seconds to load the page

Because the website is hosted on Medium.com and they make heavy use of lazy image loading.

It took ±8 secs to load for me, content was visible after 6 secs: https://i.imgur.com/6A09CDS.png

To be fair, I’ve visited websites that continue loading stuff for +3 minutes, Coinbase’s blog is lightweight in comparison.

It puzzles me a bit that they don't want to invest on.k8s expertise (at their scale, especially security-wise can be though, I guess) but at the same time they develop their own deploy system which sounds a lot like Spinnaker and they have their own secrets/config management system (Hashicorp's stack is pretty neat and battle tested).