Kubernetes is hard • Hacker News Board

Re: Kubernetes is hard

#21

Post by solatic » Mon, Mar 27, 2023, 9:42 PM UTC

endisneigh wrote: post #17

Cost aside, I wonder how far you can get with something like a managed newsql database (Spanner, CockroachDB, Vitess, etc.) and serverless. Most providers at this point offer ephemeral containers or serverless functions. Does a product focused, non infra startup even need k8s? In my honest opinion people should be using Cloud Run. It’s by far Google’s best cloud product. Anyway, going back to the article - k8s is har…

This. Greenfield products should be serverless by default. By the time you have sustained traffic to the point where you can run the numbers and think that you could save money by switching off serverless, that's a Good Problem To Have, one for which you'll have investors giving you money to hire DevOps to take care of the servers.

Re: Kubernetes is hard

#22

Post by suralind » Mon, Mar 27, 2023, 9:42 PM UTC

endisneigh wrote: post #17

Cost aside, I wonder how far you can get with something like a managed newsql database (Spanner, CockroachDB, Vitess, etc.) and serverless. Most providers at this point offer ephemeral containers or serverless functions. Does a product focused, non infra startup even need k8s? In my honest opinion people should be using Cloud Run. It’s by far Google’s best cloud product. Anyway, going back to the article - k8s is har…

I think you could push that setup far. I'm not familiar with GCP or Cloud Run, but it probably integrates nicely with other services GCP offers (for debugging, etc.).

I'd be curious to read if anybody has that setup and what scale they have.

Re: Kubernetes is hard

#23

Post by majormajor » Mon, Mar 27, 2023, 9:42 PM UTC

988747 wrote: post #5

morelisp wrote: post #4

Earlier quoted context omitted.

> Suddenly your program that wasn’t checking for errors, breaks. The memory that you didn’t manage properly becomes now a problem. Yeah, nobody deployed anything and ran it for months, even years, before Kubernetes.

Yes, they did, but did it really take less effort than running it on Kubernetes?

Definitely if they weren't running very complex services or their business tolerated an occasional maintenance window.

The particular way Kubernetes can bite you is that it makes it much easy to start with far more complex setups - not necessarily much harder than to start with a simple setup! - but then you have to maintain and operate those complex setups forever, even if you don't need them that much.

If you're growing your team and your services, having a much bigger, more complicated toolbox for people to reach into on day 1 gives you way more chance of building expensive painful tech debt.

Re: Kubernetes is hard

#24

Post by suralind » Mon, Mar 27, 2023, 9:44 PM UTC

endisneigh wrote: post #17

Cost aside, I wonder how far you can get with something like a managed newsql database (Spanner, CockroachDB, Vitess, etc.) and serverless. Most providers at this point offer ephemeral containers or serverless functions. Does a product focused, non infra startup even need k8s? In my honest opinion people should be using Cloud Run. It’s by far Google’s best cloud product. Anyway, going back to the article - k8s is har…

Regarding the second part, I totally agree, either use cloud or don't. For some reason, most companies want to be cloud-agnostic and so they stay away from things that are too difficult to migrate between cloud providers.

Re: Kubernetes is hard

#25

Post by amir734jj » Mon, Mar 27, 2023, 9:44 PM UTC

I would not use k8s unless we are convinced it will benefit us in the long run (think about constant effort that needs to put in to get things running). k8s is not magic. I would just stick with docker-compose or digital ocean for small startup. OR rent a VM on Azure OR if you really really need k8s use a managed k8s.

Re: Kubernetes is hard

#26

Post by menacingly » Mon, Mar 27, 2023, 9:44 PM UTC

birdyrooster wrote: post #18

It's not hard.

something that considerably helped my communication was to transition from "what this intelligent person said is patently wrong" to "in what sense is this intelligent person correct".

It saves me and everyone else a lot of time, because if the obvious black-and-white response were needed I probably wouldn't be having the conversation in the first place.

Re: Kubernetes is hard

#27

Post by scubbo » Mon, Mar 27, 2023, 9:44 PM UTC

> [K8s] allows dev teams to not worry about all these things; all they must do is to write a simple YAML file. More importantly, teams no longer need to ask DevOps/infra folks to add DNS entry and create a Load Balancer just to expose a service. They can do it on their own, in a declarative manner, if you have an operator do to it.

Yeah, as opposed to Cloudformation or Terraform, where you...uhhh...

Don't get me wrong, it requires work to set up your corporate infrastructure in your Favourite Cloud Provider(tm) to make those things available for developers to manage. But it takes work in k8s too - even the author says "if you have an operator to do it". Kubernetes is great for what it's great for, but these are terrible arguments in favour of it.

Re: Kubernetes is hard

#28

Post by slyall » Mon, Mar 27, 2023, 9:47 PM UTC

988747 wrote: post #5

morelisp wrote: post #4

Earlier quoted context omitted.

> Suddenly your program that wasn’t checking for errors, breaks. The memory that you didn’t manage properly becomes now a problem. Yeah, nobody deployed anything and ran it for months, even years, before Kubernetes.

Yes, they did, but did it really take less effort than running it on Kubernetes?

No it didn't. You ended up with each site doing things differently. You'd go somewhere and they would have a magical program with a cute name written by a founder that distributed traffic, scheduled jobs and did autoscaling. It would have weird quirks and nobody understood it.

Or you wouldn't have it at all. You'd have a nice simple infra and no autoscaling and deploys would be hard and involve manually copying files.

Re: Kubernetes is hard

#29

Post by brobdingnagians » Mon, Mar 27, 2023, 9:47 PM UTC

davnicwil wrote: post #20

endisneigh wrote: post #17

Cost aside, I wonder how far you can get with something like a managed newsql database (Spanner, CockroachDB, Vitess, etc.) and serverless. Most providers at this point offer ephemeral containers or serverless functions. Does a product focused, non infra startup even need k8s? In my honest opinion people should be using Cloud Run. It’s by far Google’s best cloud product. Anyway, going back to the article - k8s is har…

Or just app engine honestly. Works with docker containers so you can run the same simple stack locally as in prod. No need for more exotic serverless architectures. Generous free tier, too! Have only good things to say about it for quickly firing up a product.

Similar with fly.io, I have an application running on there and pleasantly surprised they don't even charge if the amount is under $5/month. I've been very happy with how easy it is to deploy and with the hosted Postgres. I'm using the JVM and works well; I originally played around with Elixir and was especially impressed with their feature set for it.

Re: Kubernetes is hard

#30

Post by suralind » Mon, Mar 27, 2023, 9:49 PM UTC

majormajor wrote: post #23

988747 wrote: post #5

Earlier quoted context omitted.

Yes, they did, but did it really take less effort than running it on Kubernetes?

Definitely if they weren't running very complex services or their business tolerated an occasional maintenance window. The particular way Kubernetes can bite you is that it makes it much easy to start with far more complex setups - not necessarily much harder than to start with a simple setup! - but then you have to maintain and operate those complex setups forever, even if you don't need them that much. If you're gr…

I think it may appear so because Kubernetes promotes good practices. Do logging, do metrics, do traces. That list quickly grows and while these are good practices, there's a real cost to implement them. But I wouldn't agree that Kubernetes means building tech debt - on the contrary, if you see the tech debt, k8s makes it easier to get rid of it, but that of course takes time and if you don't do it regularly that tech debt is only gonna grow.