Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Container technologies at Coinbase: Why Kubernetes is not part of our stack

blog.coinbase.com

Post reply on HN
Print view View original

251–260 of 414 posts

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#251

Post by bschwindHN »

manigandham wrote: post #248
Juliate wrote: post #244

Earlier quoted context omitted.

no downtime deployments? happened before k8s; used to do that several times with some haproxy. distributed jobs? same; nothing prevents from spawning runners with adhoc libraries and queues. managed infra? not specific to k8s

The "adhoc" part is the problem. K8S is standardized and offers high-availability, failover, logging, monitoring, load balancing, networking, service discovery, deployments, stateful services, storage volumes, batch jobs, etc. And it can run self-contained on a single machine or scale out to 1000 nodes. Why piece all of that functionality together yourself into some fragile framework instead of using a industry stand…

It can be worth it to piece things together yourself. A complex tool can also be fragile if you don't take the time to learn and understand every facet of it.

If you only need certain parts of what k8s offers, building those parts yourself can offer you more stability, control, and insight into what your application is doing.

As with anything else it varies case-by-case.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#252

Post by imtringued »

threeseed wrote: post #90
mmgutz wrote: post #64

Earlier quoted context omitted.

I'm in the same boat opting for docker-compose instead. docker-compose is much simpler to manage. Obviously it doesn't have feature parity with k8s but docker-compose does the basics well. An inexpensive VPS runs compose well with more resources at a lower cost than the managed costs of k8s.

I don't understand this comment at all. Kubernetes is designed for a cluster of nodes not a single one. Of course a single VPS is lower cost and easier to use but what happens if you want a second or third. Or any form of redundancy.

You just run scp with the compose file and run docker compose down; docker compose up

Bonus points if you just mount the compose file on an NFS share.

I've wasted enough man months on kubernetes that unless you tell me to manage 1000 nodes this approach will never cost me more time than the time I spent using and learning kubernetes.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#253

Post by asdfaoeu »

kryptk wrote: post #119
nihil75 wrote: post #78

Earlier quoted context omitted.

Odin is just Step Function flow that deploys Auto Scaling groups. It's not that complicated, and not so different from your setup.. main difference is they deploy docker images to ec2 rather than run ansible and configure a running machine.

What about resource allocation? With Kubernetes I can specify both requests and limits of cpu and memory and it will fill my nodes to match. A node autoscaler gets me more nodes if needed. I can define vertical pod autoscaler that can dynamically modify those requests and get me an emptier or bigger node if needed. I can define a horizontal pod autoscaler to keep aggregate cpu at a set target by auto-spawning more co…

Except for the vertical scaling all of that can be done with asgs/elbs.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#254

Post by rapsey »

flowerlad wrote: post #100
zaptheimpaler wrote: post #75

Earlier quoted context omitted.

Plain old linux is the alternative, which is also "learn once, use everywhere" whether its AWS EC2 or GCP Instances or nearly any machine under the sun. I don't see how k8s avoids the need to learn about cloud vendor specific tech. e.g searching "aws RDS k8s" gives me a beta github packages and a bunch of blog posts on how to configure it right. It doesn't sound like much less work than learning how to use RDS withou…

> Plain old linux is the alternative How do you run an application on a cluster of plain old linux machines? How do you do load balancing? How do you scale up and down? How do you update your app without downtime? How do you roll back easily if something goes wrong? How do you ensure all your servers are running the same version of dependencies? How do you update those dependencies? How do you replicate your environm…

Tangentially lots of those features are quite nicely handled by Erlang.

Using plain old Linux and Erlang/Elixir as a backend you can get extremely far with scaling with very little overhead and issues.

It is unfortunate that so many ignore the benefits of using Erlang as a backend technology.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#256

Post by p_l »

Juliate wrote: post #244
merb wrote: post #239

Earlier quoted context omitted.

most people probably want the following: - no downtime deployments - distributed jobs - as managed infra as possible without k8s some things would be hard.

no downtime deployments? happened before k8s; used to do that several times with some haproxy. distributed jobs? same; nothing prevents from spawning runners with adhoc libraries and queues. managed infra? not specific to k8s

Having done it myself in the past?

Setting up and taking care of k3s is much easier.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#258

Post by p_l »

bschwindHN wrote: post #251
manigandham wrote: post #248

Earlier quoted context omitted.

The "adhoc" part is the problem. K8S is standardized and offers high-availability, failover, logging, monitoring, load balancing, networking, service discovery, deployments, stateful services, storage volumes, batch jobs, etc. And it can run self-contained on a single machine or scale out to 1000 nodes. Why piece all of that functionality together yourself into some fragile framework instead of using a industry stand…

It can be worth it to piece things together yourself. A complex tool can also be fragile if you don't take the time to learn and understand every facet of it. If you only need certain parts of what k8s offers, building those parts yourself can offer you more stability, control, and insight into what your application is doing. As with anything else it varies case-by-case.

Building it yourself is also a case of "resume-driven development".

Not that there aren't good reasons or good outcomes from having done a deep dive, but just putting it there.

Re: Container technologies at Coinbase: Why Kubernetes is not part of our stack

#260

Post by jmchuster »

rudolph9 wrote: post #198
root_axis wrote: post #180

Earlier quoted context omitted.

How is a configuration.nix any different than a Dockerfile?

Dockerfiles are imperative and the order of steps changes the container image that ultimately gets built. Nix packages are declarative and idempotent. Everything is based on a functional package manager and when you configure your container via nix you get the same thing every time and can easily mix and match other packages and dependencies. A contrived example is you have a program that depends on multiple versions…

Then have your Dockerfile build from source. Many of the public layers that people build their containers upon do that for that exact reason.

If NixOS is only "generally speaking" correct because of the order it runs its scripts, then put it in a Docker container, where the whole point is to perfectly reproduce the order you run your scripts in order to minimize the errors that arise from doing otherwise. Still worried? That's why layers exist, so that every step you run you now have a static binary that you know will never shift under you, and that you can keep on building on top of.

Post reply on HN