Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

Post reply on HN
Print view View original

131–140 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#131

Post by tayo42 »

voidfunc wrote: post #128
AtlasBarfed wrote: post #125

Earlier quoted context omitted.

So you don't run any databases in those thousands of clusters? To your point, and I have not used k8s I just started to research it when my former company was thinking about shoehorning cassandra into k8s... But there was dogma around not allowing access to VM command exec via kubectl, while I basically needed it in the basic mode for certain one-off diagnosis needs and nodetool stuff... And yes, some of the floated…

> So you don't run any databases in those thousands of clusters? We do, but not of the SQL variety (that I am aware of). We have persistent key-value and document store databases hosted in these clusters. SQL databases are off-loaded to managed offering's in the cloud. Admittedly, this does simplify a lot of problems for us.

How much data? I keep hearing k8s isn't usable becasue sometimes there is to much data and it can't be moved around.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#132

Post by lkjdsklf »

valenterry wrote: post #78
lkjdsklf wrote: post #70

Earlier quoted context omitted.

We’ve been deploying software like this for a long ass time before kubernetes. There’s shitloads of solutions. It’s like minutes of clicking in a ui of any cloud provider to do any of that. So doing it multiple times is a non issue. Or automate it with like 30 lines of bash. Or chef. Or puppet. Or salt. Or ansible. Or terraform. Or or or or or. Kubernetes brings in a lot of nonsense that isn’t worth the tradeoff for…

I'm actually not using kubernetes because I find it too complex. But I'm looking for a solution for that problem and I haven't found one, so I was wondering what OP uses. Sorry, but I don't want to "click in a UI". And it is certainly not something you can just automate with 30 lines of bash. If you can, please elaborate.

> And it is certainly not something you can just automate with 30 lines of bash. If you can, please elaborate.

Maybe not literally 30.. I didn't bother actually writing it. Also bash was just a single example. It's way less terraform code to do the same thing. You just need an ELB backed by an autoscaling group. That's not all that much to setup. That gets you the two loadbalanced servers and zero downtime deploys. When you want to deploy, you just create a new scaling group and launch configuration and attach to the ELB and ramp down the old one.. Easy peasy. For the secrets, you need at least KMS and maybe secret manager if you're feeling fancy.. That's not much to setup. I know for sure AWS and azure provide nice CLIs that would let you do this in not that many commands. or just use terraform

Personally if I really cared about multi cloud support, I'd go terraform (or whatever it's called now).

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#133

Post by misswaterfairy »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I hate "cloud shit" as well, though specifically that there's a vendor-specific 'app', or terminology, or both, for everything that we've had standard terms for, for decades.

I just want a damn network, a couple of virtual machines, and a database. Why does each have to create different fancy wrappers over everything, that not even their own sales consultants, and even engineers, understand?(1)

What I do like about Docker and Kubernetes is that shifting from one cloud provider to another, or even back to on-premises (I'm waiting for the day our organisation says " is too damn expensive; those damn management consultants lied to us!!!!") is a lot easier than re-building 's bespoke shit in 's bespoke shit, or back on-premises with real tech (the right option in my opinion for anyone less than a (truly) global presence).

I do like the feel of, and being able to touch bare metal, though the 180-proof-ether-based container stuff is nice for quick, flexible, and (sometimes) dirty. Especially for experimenting when the Directors for War and Finance (the same person) say "We don't have the budget!! We're not buying another server/more RAM/another disk/newer CPUs! Get fucked!".

The other thing about Docker specifically I like is I can 'shop' around for boilerplate templates that I can then customise without having to screw around manually building/installing them from scratch. And if I fuck it up, I just delete the container and spin up another one from the image.

(1) The answer is 'vendor lock-in', kids.

(I apologise, I've had a looooooong day today.......)

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#134

Post by lkjdsklf »

jdlshore wrote: post #88
anon7000 wrote: post #59

Earlier quoted context omitted.

Tiny startups can’t afford to loose customers because they can’t scale though, right? Who is going to invest in a company that isn’t building for scale? Tiny startups are rarely trying to build projects for small customer bases (eg little scaling required.) They’re trying to be the next unicorn. So they should probably make sure they can easily scale away from tossing everything on the same server

Startups need product-market fit before they need scale. It’s incredibly hard to come by and most won’t get it. Their number one priority should be to run as many customer acquisition experiments as possible for as little as possible. Every hour they spend on scale before they need it is an hour less of runway.

while true, zero downtime deployments is... trivial... even for a tiny startup.. So you might as well do it.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#136

Post by sbstp »

paxys wrote: post #48

> Kubernetes comes with substantial infrastructure costs that go beyond DevOps and management time. The high cost arises from needing to provision a bare-bones cluster with redundant management nodes. That's your problem right there. You really don't want to be setting up and managing a cluster from scratch for anything less than a datacenter-scale operation. If you are already on a cloud provider just use their mana…

Most control planes are not free anymore, they cost like 70$/mo on AWS & GCP. Used to be a while back.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#137

Post by oofbey »

paxys wrote: post #48

> Kubernetes comes with substantial infrastructure costs that go beyond DevOps and management time. The high cost arises from needing to provision a bare-bones cluster with redundant management nodes. That's your problem right there. You really don't want to be setting up and managing a cluster from scratch for anything less than a datacenter-scale operation. If you are already on a cloud provider just use their mana…

If you do find yourself wanting to create a cluster by hand, it's probably because you don't actually need lots of machines in the "cluster". In my experience it's super handy to run tests on a single-node "cluster", and then k3s is super simple. It takes something like 8 seconds to install k3s on a bare CI/CD instance, and then you can install your YAML and see that it works.

Once you're used to it, the high-level abstractions of k8 are wonderful. I run k3s on raspberry pi's because it takes care of all sorts of stuff for you, and it's easy to port code and design patterns from the big backend service to a little home project.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#138

Post by photonthug »

stickfigure wrote: post #71
ants_everywhere wrote: post #42

People talk about Kubernetes as container orchestration, but I think that's kind of backwards. Kubernetes is a tool for creating computer clusters. Hence the name "Borg" (Kubernetes's grandpa) referring to assimilating heterogeneous hardware into a collective entity. Containers are an implementation detail. Do you need a computer cluster? If so k8s is pretty great. If you don't care about redundancy and can get all t…

K8s doesn't help you solve your geographical region problem, because the geographical region problem is not running appserver instances in multiple regions. Almost any PaaS will do that for you out of the box, with way less fuss than k8s. The hard part is distributing your data. Less overhead than writing your own glue code, less overhead than learning Kubernetes, is just use a PaaS like Google App Engine, Amazon Ela…

> just use a PaaS like Google App Engine, Amazon Elastic Beanstalk, Digital Ocean App Platform, or Heroku.

This is the right way for web most of the time, but most places will choose k8s anyway. It’s perplexing until you come to terms with the dirty secret of resume driven development, which is that it’s not just junior engs but lots of seniors too and some management that’s all conspiring to basically defraud business owners. I think the unspoken agreement is that Hard work sucks, but easy work that helps you learn no transferable skills might be worse. The way you evaluate this tradeoff predictably depends how close you are to retirement age. Still, since engineers are often disrespected/discarded by business owners and have no job security, oaths of office, professional guilds, or fiduciary responsibility.. it’s no wonder things are pretty mercenary out there.

Pipelines are as important as web these days but of course there are many options for pipelines as a service also.

K8s is the obviously correct choice for teams that really must build new kinds of platforms that have many diverse kinds of components, or have lots of components with unique requirements for coupling (like say “scale this thing based on that other thing”, but where you’d have real perf penalties for leaving the k8s ecosystem to parse events or whatever).

The often mentioned concern about platform lock in is going to happen to you no matter what, and switching clouds completely rarely happens anyway. If you do switch, it will be hard and time consuming no matter what.

To be fair, k8s also enables brand new architectural possibilities that may or may not be beautiful. But it’s engineering, not art, and beautiful is not the same as cheap, easy, maintainable, etc.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#139

Post by fragmede »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'll let you in on the joke. The joke is the demand for 100% availability and instant gratification. we're making services where anything less than 4 nines, which is 5 minutes month, is deemed unacceptable. three nines is 10 minutes a week. two nines is 15 minutes a day. there are some things that are important enough that you can't take a coffee break and wait for, but Kubernetes lets you push four nines of availability, no problem. Kubernetes is solving for that level of availability, but my own body doesn't have anything near that level of availability. demanding that from everything and everyone else is what pushes for Kubernetes level of complexity.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#140

Post by philbo »

We migrated to Cloud Run at work last year and there were some gotchas that other people might want to be aware of:

1. Long running TCP connections. By default, Cloud Run terminates inbound TCP connections after 5 minutes. If you're doing anything that uses a long-running connection (e.g. a websocket), you'll want to change that setting otherwise you will have weird bugs in production that nobody can reproduce in local. The upper limit on connections is 1 hour, so you will need some kind of reconnection logic on clients if you're running longer than that.

Ref: https://cloud.google.com/run/docs/configuring/request-timeou...

2. First/second generation. Cloud Run has 2 separate execution environments that come with tradeoffs. First generation emulates Linux (imperfectly) and has faster cold starts. Second generation runs on actual Linux and has faster CPU and faster network throughput. If you don't specify a choice, it defaults to first generation.

Ref: https://cloud.google.com/run/docs/about-execution-environmen...

3. Autoscaling. Cloud Run autoscales at 60% CPU and you can't change that parameter. You'll want to monitor your instance count closely to make sure you're not scaling too much or too little. For us it turned out to be more useful to restrict scaling on request count, which you can control in settings.

Ref: https://cloud.google.com/run/docs/about-instance-autoscaling

Post reply on HN