Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Container technologies at Coinbase: Why Kubernetes is not part of our stack

blog.coinbase.com

Post reply on HN
Print view View original

41–50 of 414 posts

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#41

Post by gen220 »

nemothekid wrote: post #3

One thing that is regrettable about K8s winning the orchestration wars so remarkably, is that it pretty much killed all other solutions. Swarm is dead, Nomad doesn't seem like it has much community support and Mesos feels like it's on life support. Mesos still has a lot of people working on it however, but the perception feels different. Personally I've found Mesos much easier to manage, secure, and operate than k8s.…

We‘ve been mostly happy users of Mesos, Apache aurora, and consul. It works pretty well for us (200+ engineers). We have maybe 5 people dedicated to keeping it all alive, and they’d be able to maintain the pieces of it that we use. Aurora configuration kinda sucks, but I think job configurations might just suck in general.

That being said, it has been concerning to us to see the big institutional players move off of mesos and aurora. We wouldn’t like to be active maintainers, but we could.

I think that’s the main difference between Mesos and K8s: with K8s, we wouldn’t want to maintain it, and we wouldn’t be able to (since it’s so large). Somehow mesos and aurora feel more manageable.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#42

Post by flowerlad »

dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

> If you want k8s, you really do need people that know how to maintain it on a more or less full time schedule.

What is the alternative to k8s that does not need people to have any technical knowledge?

To me Kubernetes is extremely attractive because it helps me avoid learning cloud vendors' proprietary technologies. K8s is learn once, use everywhere, which is fantastic.

I am a 1-person venture doing everything from JavaScript/React to maintaining backend infra, and I couldn't have done it without k8s.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#43

Post by siberianbear »

ur-whale wrote: post #16

Given Coinbase's reliability track record [1][2][3], I'm not entirely sure why I'd listen to devops "wisdom" coming from their corner. [1] https://www.reddit.com/r/CoinBase/comments/gh3b5t/coinbase_c... [2] https://www.reddit.com/r/Bitcoin/comments/6gtwyi/every_singl... [3] https://www.reddit.com/r/CryptoCurrency/comments/ggr80l/i_do...

They've having some growing pain. It's normal in a company growing at Coinbase's pace.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#45

Post by michaeldwan »

nemothekid wrote: post #3

One thing that is regrettable about K8s winning the orchestration wars so remarkably, is that it pretty much killed all other solutions. Swarm is dead, Nomad doesn't seem like it has much community support and Mesos feels like it's on life support. Mesos still has a lot of people working on it however, but the perception feels different. Personally I've found Mesos much easier to manage, secure, and operate than k8s.…

We're using Nomad to manage a large fleet of firecracker vms at fly.io. It's not as robust as k8s, but I think that's a feature. It's well documented, extensible, and predictable. Not a big community, but hashicorp folks are responsive on GitHub.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#46

Post by lifeisstillgood »

flowerlad wrote: post #42
dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

> If you want k8s, you really do need people that know how to maintain it on a more or less full time schedule. What is the alternative to k8s that does not need people to have any technical knowledge? To me Kubernetes is extremely attractive because it helps me avoid learning cloud vendors' proprietary technologies. K8s is learn once, use everywhere, which is fantastic. I am a 1-person venture doing everything from…

Firstly tell me more about your 1-person full-stack venture, but second how comes you, with barely any time for sitting down can use k8s happily but it falls over for others. I am struggling to see truth amount the comments here :-(

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#47

Post by croh »

thrwn_frthr_awy wrote: post #22
dvt wrote: post #11

> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…

> This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. I've experienced this as well. At the last large company I worked at we had a…

Been there. Simple python django app. Max 20 APIs. Expecting 100 requests per month. But POs and directors wanted shiny DevOps tools. Problems with typical MNCs. Every quarter manager/director comes with some new hype.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#48

Post by yahyaheee »

darkwater wrote: post #19

It puzzles me a bit that they don't want to invest on.k8s expertise (at their scale, especially security-wise can be though, I guess) but at the same time they develop their own deploy system which sounds a lot like Spinnaker and they have their own secrets/config management system (Hashicorp's stack is pretty neat and battle tested).

Works great! ... till it doesn’t

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#49

Post by mickeyben »

Thanks for the article. I've been in tech for the past 10 years, working in or around devops teams for the most part but I don't get all the fuss about k8; yes it's an amazing tool doing a lot more than any other.

But it has a big learning curve and setup & maintenance are very costly. I don't understand why most orgs are moving to k8 considering this. When talking to my peers I often see numbers like 6 months to 2 years full migration with a very small added value - at least for 90%+ of the companies using it.

It usually boils down to attracting talents and keeping them excited trying the new shit.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#50

Post by d0m »

Quick shout-out toAptible (Heroku for HIPAA) which have been amazing for our healthcare startup.

Still, I honestly feel that we've taken a step back in the industry where we went from ec2 --> heroku --> k8s. But I know there are many people working hard to create the next infrastructure so we don't have to deal with containers and all that re-inventing dev-ops non-sense for every project.

Post reply on HN