Live data from Hacker News

Container technologies at Coinbase: Why Kubernetes is not part of our stack

blog.coinbase.com

231–240 of 414 posts

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#231

Earlier quoted context omitted.

> If you want k8s, you really do need people that know how to maintain it on a more or less full time schedule. What is the alternative to k8s that does not need people to have any technical knowledge? To me Kubernetes is extremely attractive because it helps me avoid learning cloud vendors' proprietary technologies. K8s is learn once, use everywhere, which is fantastic. I am a 1-person venture doing everything from…

Firstly tell me more about your 1-person full-stack venture, but second how comes you, with barely any time for sitting down can use k8s happily but it falls over for others. I am struggling to see truth amount the comments here :-(

Might also be the amount of experience a person has.

I have done Softwaredevelopment for ~15 years, swiched now to infrastructure and build a gke cluster. It was awesome, very logical, nice and easy to use.

Now i read stories from coinbase and don't get it.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#232

Earlier quoted context omitted.

One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own lar…

It amazing to me how many people associate this with kube. You’ve had this for years in numerous ways, the only difference is some yaml and marketing.

What other solution is cloud agnostic as well as has an easy local dev story?

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#233

I work for a mid size company with 30-40 engineers managing 20-30 very diverse apps in terms of scale requirements and architectural complexity. It took our devops team (4-5 people) probably 18 months to learn and fully migrate all our apps to Kubernetes. The upfront cost was massive, but nowadays the app teams own their deployments, configurations, SLA's, monitors, and cost metrics. Introducing Kubernetes into our o…

One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own lar…

Isn't that the benefit of cloud?

Maybe I'm spoiled by GCP and it's not the same in other providers; but I can have a brand new debian VM configured and operational in less than 10s (less the time that I run my own startup script).

Debian machines spawn with incredible speed; not much slower (if at all slower) than a new container.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#234
post #131

Earlier quoted context omitted.

I work on a 2-person project and decided to go with kubernetes (through digitalocean) for the cluster. I am managing everything with terraform and I don't have any big problems. I like that I can write everything as terraform manifests, have it diffed on git push and applied to prod if I want to. Sure it had a learning curve but now I just describe my deployments and k8s does the rest, which then reflects back on dig…

if you can run everything on a $10 vm...do you really need k8 ?

You do if you do a resume driven development.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#235
Question: some of the security/management concerns cannot be addressed via AWS Fargate?

The bottom line is: K8s is awsome but it is complex. It is a Google-needs-oriented software. Sadly average project will not have that complexity you find in Google Plex.

I can run Oracle Database in 100MB of RAM...you need 4GB at least to run a K8s node... a costly option. Below ten servers I find no point in K8s: I highly suggest docker swarm.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#236

At this point I’m of the belief that adopting k8s at most companies should result in shareholder/investor lawsuits for the near-criminal waste of corporate resources.

I'm not sure why.

My personal experience is very good and i'm looking forward to a bright future.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#237
post #74

Earlier quoted context omitted.

I'd agree that k8s has a lot of functionality built-in, another important thing to realise is what k8s doesn't do. In addition to the well-known integration points (Container Runtime/Network/Storage Interfaces), there's things like the lack of a good built-in user authentication mechanism with Kubernetes, which means you pretty much always need some external authentication service for your clusters. That's not too ba…

> That's not too bad if your on one of the big managed providers (GKE/AKS/EKS) but can get complex for people who want to deploy on-prem. Go spin up Keycloak, join it to your user-directory of choice (or not and just use the internal directory), configure it as your authentication provider, done.

Right so in addition to the complexity of running k8s (which is the general point of the post) you now have to learn about OAuth servers and LDAP integration.

In many corporates you also now have the challenges of cross-team/department work, for the k8s team to work with the AD team to get it setup.

And still that won't get you away from the problem that without a first class user / group object in k8s people often end up running into problems with JML processes over time and mismatch between AuthN and AuthZ...

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#239
post #131

Earlier quoted context omitted.

I work on a 2-person project and decided to go with kubernetes (through digitalocean) for the cluster. I am managing everything with terraform and I don't have any big problems. I like that I can write everything as terraform manifests, have it diffed on git push and applied to prod if I want to. Sure it had a learning curve but now I just describe my deployments and k8s does the rest, which then reflects back on dig…

if you can run everything on a $10 vm...do you really need k8 ?

most people probably want the following: - no downtime deployments - distributed jobs - as managed infra as possible

without k8s some things would be hard.

Post reply on HN