Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Container technologies at Coinbase: Why Kubernetes is not part of our stack

blog.coinbase.com

Post reply on HN
Print view View original

211–220 of 414 posts

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#211

Post by amerine »

ec109685 wrote: post #184
DevKoala wrote: post #154

I work for a mid size company with 30-40 engineers managing 20-30 very diverse apps in terms of scale requirements and architectural complexity. It took our devops team (4-5 people) probably 18 months to learn and fully migrate all our apps to Kubernetes. The upfront cost was massive, but nowadays the app teams own their deployments, configurations, SLA's, monitors, and cost metrics. Introducing Kubernetes into our o…

One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own lar…

It amazing to me how many people associate this with kube. You’ve had this for years in numerous ways, the only difference is some yaml and marketing.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#212

Post by rumanator »

redis_mlc wrote: post #203
rumanator wrote: post #195

Earlier quoted context omitted.

Your comment makes no sense at all. Kubernetes is not equivalent to running nginx on Ubuntu. This comparison is simply too absurd to the point I really doubt you have any idea of what Kubernetes is designed to do and why people use it. I mean, who in their right mind would even describe a cluster of VMs and/or COTS hardware running with a VPN and designed to automatically run and manage the lifecycle of containers as…

rumanator: the mistake is yours. Please read the parent post again, since you missed any nuance. I agree with said parent. You can go a long way with ASGs, iptables and yum install - literally the entire first decade of your startup. In larger companies, just the politics introduced with managing k8s by another dept. is mind-numbing.

No, the assertion that nginx on ubuntu is equivalent to kubernetes os mind-numbingly wrong, for starters for being entirely and completely oblivious to containers. The comparison isn't even wrong: it simply makes no sense at all.

And no, being able to run software is not equivalent to Kubernetes. It's not even in the same ballpark of the discussion. You are entirely free to go full Amish on your pet projects but let's not pretend that managing a pet server brings any operational advantage over, say, just being able to seamlessly deploy N versions of your app with out-of-the-box support for blue/green deployments and with the help of a fully versioned system that allows to undo and resume operating with a past configuration. You don't do that by flaunting iptables and yum, do you?

If anyone needs to operate a cluster then they need tooling to manage and operate a cluster. Your iptables script isn't it.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#213

Post by amerine »

whatsmyusername wrote: post #185
AaronM wrote: post #112

Earlier quoted context omitted.

We've been using EKS since it went GA. We haven't had a single control plane outage that I am aware of.

I'm sure it works fine if you can get it running (documentation didn't work when I played with it). I'm referring more to the $200/month it was per control plane. That to me is a product they offer because someone else is offering it, but they don't want you to actually use it.

I take it the isolated $200/control plane is too expensive. What do you think the right price point is for that isolation?

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#214

Post by tannhaeuser »

I wonder if mesos with it's somewhat more tidied up infrastructure and codebase and less moving parts could make it as a k8s replacement, if you absolutely must run Linux container workloads (though I understand mesos can run a anything not just Docker-/runc-like images).

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#215

Post by redis_mlc »

rumanator wrote: post #212
redis_mlc wrote: post #203

Earlier quoted context omitted.

rumanator: the mistake is yours. Please read the parent post again, since you missed any nuance. I agree with said parent. You can go a long way with ASGs, iptables and yum install - literally the entire first decade of your startup. In larger companies, just the politics introduced with managing k8s by another dept. is mind-numbing.

No, the assertion that nginx on ubuntu is equivalent to kubernetes os mind-numbingly wrong, for starters for being entirely and completely oblivious to containers. The comparison isn't even wrong: it simply makes no sense at all. And no, being able to run software is not equivalent to Kubernetes. It's not even in the same ballpark of the discussion. You are entirely free to go full Amish on your pet projects but let'…

> No, the assertion that nginx on ubuntu is equivalent to kubernetes os mind-numbingly wrong

Read the actual post, instead of projecting your nonsense.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#216

Post by gigatexal »

arcticfox wrote: post #141
dvcrn wrote: post #131

Earlier quoted context omitted.

I work on a 2-person project and decided to go with kubernetes (through digitalocean) for the cluster. I am managing everything with terraform and I don't have any big problems. I like that I can write everything as terraform manifests, have it diffed on git push and applied to prod if I want to. Sure it had a learning curve but now I just describe my deployments and k8s does the rest, which then reflects back on dig…

I'm also on a 2-person project on DigitalOcean k8s, also very happy. K8s is kind of messy compared to Heroku, which I don't love, but is also way more powerful and can be more secure. I don't know what I'd use instead of it, exactly as you said. Also, we run a VPC-only K3s node for some simple internal tools that works great as well.

I’m a bit dubious in the Heroku-is-less-secure-than-k8s claims

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#218

Post by santamarias »

jillesvangurp wrote: post #210
dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

It's a pity that docker swarm did not make it. It wasn't perfect but it was a lot simpler to setup and manage than kubernetes. If you can get away with it, vanilla docker hosts running docker compose provide most of the same benefits with a fraction of the cost. For most startups, that's a great way to avoid getting sucked into a black hole of non value adding devops activity. You lose some flexibility but vanilla ub…

> It's a pity that docker swarm did not make it.

Sorry I do not understand that statement, in my naive opinion Docker Swarm seems to be a thing. Care to elaborate, please?

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#219

Post by gigatexal »

Imo: the best way to run K8s is via some managed provider. Sure you can do it the hardway a la Kelsey Hightower but in production and if you’re not staffed with k8s experts I’d rather give that to a provider and focus on what I know — the code and biz logic.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#220

Post by bule0bird »

dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

I had a similar problem, setup a small cluster and it worked fine until next upgrade and nightmare repeated on each upgrade. I realize spending most of my time on fixing k8s issues rather than doing any valuable work
Post reply on HN