Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Container technologies at Coinbase: Why Kubernetes is not part of our stack

blog.coinbase.com

Post reply on HN
Print view View original

131–140 of 414 posts

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#131

Post by dvcrn »

dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

I work on a 2-person project and decided to go with kubernetes (through digitalocean) for the cluster. I am managing everything with terraform and I don't have any big problems. I like that I can write everything as terraform manifests, have it diffed on git push and applied to prod if I want to.

Sure it had a learning curve but now I just describe my deployments and k8s does the rest, which then reflects back on digitalocean. If I need more power for my cluster, I increase the nodes through digitalocean and k8s automatically moves my containers around how it deems fit.

I used normal blue/green deployments on self-managed VMs in the past, then worked with beanstalk, heroku, appengine and I much prefer k8s. Yes it's easier on heroku, but try to run 2-3 different containers on the same dyno for dev to keep cost down. On k8s I can run my entire stack on one single small digitalocean $10 VM if I wanted to.

I wouldn't even know what I else could pick that gives me equal flexibility and power?

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#132

Post by [deleted] »

flowerlad wrote: post #100
zaptheimpaler wrote: post #75

Earlier quoted context omitted.

Plain old linux is the alternative, which is also "learn once, use everywhere" whether its AWS EC2 or GCP Instances or nearly any machine under the sun. I don't see how k8s avoids the need to learn about cloud vendor specific tech. e.g searching "aws RDS k8s" gives me a beta github packages and a bunch of blog posts on how to configure it right. It doesn't sound like much less work than learning how to use RDS withou…

> Plain old linux is the alternative How do you run an application on a cluster of plain old linux machines? How do you do load balancing? How do you scale up and down? How do you update your app without downtime? How do you roll back easily if something goes wrong? How do you ensure all your servers are running the same version of dependencies? How do you update those dependencies? How do you replicate your environm…

[deleted]

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#133

Post by daze42 »

manigandham wrote: post #99
dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

You can't make the complexity disappear. Kubernetes just offers a very standardized, stable and relatively polished way to handle it. The other alternatives, like running your own orchestration system, are just as complex if not more so, although you might be more familiar with it since you built it. That being said, many companies don't need any of it in the first place for their scale, and that's probably the bigge…

It's impossible to make essential complexity disappear but it is certainly possible to reduce incidental complexity. Most software is much more complex than it needs to be and Kubernetes is no exception.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#134

Post by 013a »

dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

> If you want k8s, you really do need people that know how to maintain it on a more or less full time schedule.

I think this is roughly similar to saying "if you want linux, you need people that know how to maintain it." Which is to say, you can create an architecture where this is absolutely true, but it doesn't need to be true.

The big issue with K8S right now isn't K8S; its that there aren't (big, well established) solutions like Heroku or Zeit or whatever, for K8S, where you don't need to worry about "the cluster", just like those solutions don't make you worry about Linux. K8S really is two parts; the API and the Cluster. The API is the more valuable of the two.

And, you know, maybe it won't ever get there. Heroku and Zeit solve strikingly similar problems to K8S. Maybe K8S just is a platform like that, but for enterprises who want to home-grow, and maybe most companies shouldn't worry about it. But I think the platform, and thus the community, simply needs more time to figure out where it makes sense.

Most companies shouldn't touch K8S. You'll probably regret it. But, to your second point: AWS literally has nothing beyond EKS/ECS + Fargate which approaches a Heroku-like service. Beanstalk is supposed to be that, but its really just a layer on-top of EC2 which doesn't touch the "ultra low maintenance" of a Heroku, or Zeit, or App Engine. So if you're on AWS, and you want to use their other excellent managed services, you either go outside AWS, or you'll go EKS, or you'll end up trying to in-house something even worse.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#135

Post by manigandham »

yjftsjthsd-h wrote: post #124
manigandham wrote: post #99

Earlier quoted context omitted.

You can't make the complexity disappear. Kubernetes just offers a very standardized, stable and relatively polished way to handle it. The other alternatives, like running your own orchestration system, are just as complex if not more so, although you might be more familiar with it since you built it. That being said, many companies don't need any of it in the first place for their scale, and that's probably the bigge…

I'd say it the other way; k8s has to do all the things that another management system would have to do, but sufficiently tied together that you can't ease into it as needed.

I guess that depends primarily on whether you're installing and operating K8S yourself. Use something like GKE and it's a very seamless experienced, with AKS getting pretty good and the rest being rather crummy.

Once you have a managed cluster, the deploying apps is fairly easy. A single container/pod is a 1-liner and you can work your way up from there.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#136

Post by dvcrn »

croh wrote: post #47
thrwn_frthr_awy wrote: post #22

Earlier quoted context omitted.

> This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. I've experienced this as well. At the last large company I worked at we had a…

Been there. Simple python django app. Max 20 APIs. Expecting 100 requests per month. But POs and directors wanted shiny DevOps tools. Problems with typical MNCs. Every quarter manager/director comes with some new hype.

If it's a hosted k8s, all you'd do is containerize your application, then create a deployment that pulls that container and exposes it through a loadbalancer. The container is the same if you'd push it to herokus container registry or beanstalk

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#137

Post by ilaksh »

My feeling is that I just want AWS Lambda functions to allow for larger packages. And then clusters and scaling and OS updates etc. are Amazon's problem.

I would rather just be able to do that than get into ECS/Fargate, much less K8s. It seems like all of that stuff is just adding more complexity for me.

Of course none of my projects are gigantic or need to be highly secure.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#138

Post by harpratap »

dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

I have said this multiple times in the past (https://news.ycombinator.com/item?id=23361176 https://news.ycombinator.com/item?id=23243626) and will say it again - You (business logic developers) are not meant to use K8s directly, you are supposed to use a PaaS built on top of K8s (like Cloud Functions, Lambda etc)

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#139

Post by erulabs »

dpenguin wrote: post #15

Can fully relate to the author. We have been struggling with effective k8s at our company for 2 years now. Too much to learn to get your first service in production. You will end up writing wrappers after wrappers that make you think “we can’t be the first guys to solve this” and after googling, you find that every issue you find is described as “just” one of the downsides of using k8s and here’s how we overcame it.…

> I wish someone can make a deployment orchestrator for your private DC that is as simple to use as Heroku is.

This is what I'm working on, although it is built on top of Kubernetes! We feel Kube suffers from a disconnect between the insane complexity requirements of enterprise deployments and what most coders and businesses actually need - which is exactly what you said: a consistent Heroku-ish experience on their own hardware or whatever cloud is convenient. Kube needs what Git needed - a GitHub. Feel free to reach out to me (email in profile), I'd be happy to chat your ear off about it!

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#140

Post by snuxoll »

threeseed wrote: post #95
snuxoll wrote: post #74

Earlier quoted context omitted.

> That's not too bad if your on one of the big managed providers (GKE/AKS/EKS) but can get complex for people who want to deploy on-prem. Go spin up Keycloak, join it to your user-directory of choice (or not and just use the internal directory), configure it as your authentication provider, done.

Or use Dex: https://github.com/dexidp/dex Which has the advantage of not needing any external databases.

That works too depending on your requirements. Either way, authentication is not a hard problem to solve.
Post reply on HN