Kubernetes is a red flag signalling premature optimisation • Hacker News Board

Re: Kubernetes is a red flag signalling premature optimisation

#391

Post by philliphaydon » Mon, Jul 04, 2022, 3:47 PM UTC

boloust wrote: post #325

philliphaydon wrote: post #226

Earlier quoted context omitted.

> Setting up a new application on IIS or a new web server to scale is a mare. I disagree. With octopus deploy I can add a step, set a package and hostname, and press a button, and have a new deployment of a new api or website in IIS pushed out to how ever many servers currently exist in a few minutes. There are many ways to manage deploying services, and scaling, without K8S or containers in general. While I could se…

That is very interesting, I'd love to know more about how this works. - How does it decide which servers to deploy to? - Does it scale up the number of servers if the deployment is too large? - What happens when a server dies?

Well I guess it all depends on your setup.

Servers get have a tentacle which has an environment and tag. So you could say a server is test environment and tagged with main-web-app. While another server is production with the same tag. You promote a package from test to production. You configure your setup to say a package is deployed to a server tagged main-web-app.

Octopus is an orchestration tool so isn’t responsible for scaling up. But in AWS with an autoscale group you can configure Octopus to auto detect a new server tag it and deploy. Part of the deployment would add it to the load balancer.

As a side note you can deploy containers using octopus too. Tho I’ve never found a reason to use containers in production yet.

Re: Kubernetes is a red flag signalling premature optimisation

#392

Post by lebowen » Mon, Jul 04, 2022, 3:47 PM UTC

MaKey wrote: post #301

lebowen wrote: post #190

Earlier quoted context omitted.

I'm aware of these products, but do not fit my use case. I need to run some services on premises and have set up a self hosted Kubernetes instance on a physical server in a rack. It could be overkill and maybe I could use something like Docker Swarm. Apart from this I am unsure what I can use that isn't K8s to orchestrate my containers on site.

Which features do you need that docker compose can't meet?

Rolling restart for zero downtime deployments.

Re: Kubernetes is a red flag signalling premature optimisation

#393

Post by mirceal » Mon, Jul 04, 2022, 3:49 PM UTC

cjalmeida wrote: post #218

wodenokoto wrote: post #155

It’s kinda curious that the HN sentiment in the comments has lately switched from “kubernetes is hell” to “kubernetes is quite useful and well worth it” To me, kubernetes is the kind of giant, complicated, hard to understand software that is ripe for HN disdain.

Modern CPUs are giant, complicated pieces of hardware. But they’re useful, and have a well defined interface that you can interact with it at a higher level. Same with k8s. People conflated running k8s from scratch, with administrating. The former is very hard, you should buy it from a cloud provider. The latter is IMO not much harder than VMs at small scale and much simpler at you grow bigger.

Yes, let's compare CPUs that took literally decades to mature and there are a handful of companies that can design and manufacture them to K8s.

We don't want to run k8s from scratch? Let's buy the managed version in the cloud?

It sounds like an argument for containers.

I mean, at that point why not use a simpler abstraction, like a VM. Or just use functions.

Re: Kubernetes is a red flag signalling premature optimisation

#394

Post by vbezhenar » Mon, Jul 04, 2022, 3:51 PM UTC

bheadmaster wrote: post #159

Chilinot wrote: post #131

Earlier quoted context omitted.

Could you give some examples of these network distributed OS's that are going full steam ahead? Also interested in why you think those are less clunky than k8s. :)

Plan9 has basically invented the idea of namespaces, which are fundamental building blocks of containers. Plan9 also invented 9P file protocol which is used in WSL2. Plan9 is the ultimate distributed OS. Running programs on different computers is as easy as mounting a remote /dev/cpu file to your process' namespace. Virtual desktop is as easy as mounting /dev/draw. And so on. Many such complicated things that require…

Distributed OS hides distributed nature from the application. Kubernetes does not try to hide it. It allows developer to embrace it. Hiding network latency is leaky abstraction. Kubernetes does it right. It does not pretend that there's no network between your services. It's the other way around: there's always network between your services. Sometimes it's loopback network, sometimes it's real network, but you're aware of it.

Re: Kubernetes is a red flag signalling premature optimisation

#395

Post by javajosh » Mon, Jul 04, 2022, 3:56 PM UTC

gizzlon wrote: post #367

javajosh wrote: post #358

Earlier quoted context omitted.

What's your plan when (not if) Google deprecates GAE?

Not OP, but App Engine have actually been around for a long time. Also, there's no inherent lock-in, you can basically just deploy it somewhere else. Data is where the lock-in lies. Moving can be hard if you use proprietary databases. Can still be worth it.

>App Engine have actually been around for a long time

That means very little, I hope you realize. Reader, Voice, Chat, etc.[0] were all around a long time.

>Also, there's no inherent lock-in

GAE has plenty of proprietary APIs you can depend on. Whether or not you do is up to the programmer.

0 - A comment below notes that voice and chat aren't deprecated yet. Voice was announced deprecated, and google has had so many chat apps I'm not sure which ones are gone. Anyway, here is a more complete list of things Google has abandoned: https://killedbygoogle.com/

Re: Kubernetes is a red flag signalling premature optimisation

#396

Post by p_l » Mon, Jul 04, 2022, 3:56 PM UTC

sudarshnachakra wrote: post #282

p_l wrote: post #268

Earlier quoted context omitted.

It's because OTP does not integrate with anything not running on Erlang VM, and k8s instead derives from different family tree of general language-independent schedulers/environments.

Language independence is not a trait of k8s it's an artifact of docker packaging java/c++/perl/python/go/rust etc. as an arch dependent image. TBH I find k8s support for languages other than Golang pretty poor (there have been attempts to get java into k8s by redhat with native-image, but it seems to have not made it big).

Language independence is a trait of k8s in the sense that none of its interfaces are in any way specific to a language - the most restrictive in that are the few APIs that are based on gRPC, because state of gRPC libraries is still poor in some places.

Unless you want to embed another language in-process of some k8s components, but the need to do that is disappearing as things are moved out of process.

Re: Kubernetes is a red flag signalling premature optimisation

#397

Post by charlie0 » Mon, Jul 04, 2022, 3:57 PM UTC

ryanbrunner wrote: post #312

okamiueru wrote: post #106

Earlier quoted context omitted.

I don't think their advice about not using it in a startup is correct either. You just need to somewhat know what you're doing. I know of such a case, where a single engineer could leverage the helm chart open source community, and set up a scalable infrastructure, with prometheus, grafana, worker nodes that can scale independently of web service, a CI/CD pipeline that can spin up complete stacks with TLS automated t…

The thing is, unless using those technologies was somehow core to what the single engineer was trying to, it might be technically impressive but might not have actually provided value for users. Users don't really care if you have a really impressive stack with cool technologies if it doesn't offer anything more than a couple of web servers and a DB server.

Right on. Previous devs at co I joined wanted to play DevOps cowboys. They used Ansible scripts to spin up various AWS services costing the company over 100K/yr.

New lead came in, got rid of that crap by using 3rd party services to spin up infrastructure. Got a load balancer, a few VMs + DB. Reduced the cost down by 85% and greatly simplified the entire stack.

I learned a really valuable lesson without having to make that mistake myself.

I understand why people get excited about tooling. It's cool to learn new things and automate stuff away. I'm prone to that myself and do this on my own server when I get that itch.

Having said that, it's wrong to foist this stuff onto an unsuspecting company where the owners don't know any better about tech, that's why they hire other people to do that for them and seeing that just left a bad taste in my mouth for overcomplicated setups.

I get that SV is different, that's why tools like K8 are made and I would jump on those tools on a heartbeat as needed.

But for other smaller businesses, the truth is they just need a boring monolothic load balanced app with a few VMs and a DB sprinkled with 3rd party services for logging or searching or other stuff not core to the business.

Re: Kubernetes is a red flag signalling premature optimisation

#398

Post by goodpoint » Mon, Jul 04, 2022, 4:00 PM UTC

bheadmaster wrote: post #159

Chilinot wrote: post #131

Earlier quoted context omitted.

Could you give some examples of these network distributed OS's that are going full steam ahead? Also interested in why you think those are less clunky than k8s. :)

Plan9 has basically invented the idea of namespaces, which are fundamental building blocks of containers. Plan9 also invented 9P file protocol which is used in WSL2. Plan9 is the ultimate distributed OS. Running programs on different computers is as easy as mounting a remote /dev/cpu file to your process' namespace. Virtual desktop is as easy as mounting /dev/draw. And so on. Many such complicated things that require…

This! Real innovation is doing something without introducing unnecessary complexity and enormous codebases.

Re: Kubernetes is a red flag signalling premature optimisation

#399

Post by decidertm » Mon, Jul 04, 2022, 4:03 PM UTC

There is a lot to agree on in this article. Though teams should be able to choose whatever stack (languages, frameworks, databases, queues) they want. What’s important is there is a coherent way to deploy, build, operate, scale, observe these services, without relying on one or two team members. Getting a cluster setup and a few deployments is pretty simple, however scaling that cluster, setting up secure runtime (network policies, mTLS, Kata, KVM, gVisor), prometheus, CI/CD, preview, staging and production environments is a huge investment (personnel, cloud costs, delay to production delivery).

Teams should be able to benefit from Kubernetes and the surrounding cloud-native ecosystem without directly consuming or modifying it. You don’t need to reinvent the wheel, and that’s what you could say a lot of platform teams are doing today.

That’s what we’re working on at https://northflank.com a next-generation deployment platform using Kubernetes either in a secure multi-tenant environment, traditional PaaS or deployed directly into your GKE, EKS and AKS clusters. (disclaimer: Northflank co-founder)

Re: Kubernetes is a red flag signalling premature optimisation

#400

Post by gjulianm » Mon, Jul 04, 2022, 4:03 PM UTC

wronglyprepaid wrote: post #368

gjulianm wrote: post #75

Earlier quoted context omitted.

> I guess redundancy is not really a thing then? Most of the time it really doesn't need to be. In the end what you care about is uptime and cost. A redundant solution doesn't have a perfect uptime just because it's redundant, in fact sometimes it might have even less uptime because of failures in the redundancy mechanism. Of course if you need to be always up it might be worth it. But for a lot of situations some do…

> A redundant solution doesn't have a perfect uptime just because it's redundant, in fact sometimes it might have even less uptime because of failures in the redundancy mechanism I'm fairly sure that google does a better job keeping cloud spanner and cloud run working, and their redundancy mechanisms working, than whoever runs your single linux box will do. > most of the time it's better to be on a simpler setup that…

> I'm fairly sure that google does a better job keeping cloud spanner and cloud run working, and their redundancy mechanisms working, than whoever runs your single linux box will do.

Most of the uptime loss won't come from your provider but from your applications and configuration. If you use Cloud Run and mess up the configuration for the redundancy, you'll still have downtime. If your application doesn't work well with multiple instances, you'll still have downtime.

> Keeping a whole linux box running is more complicated and requires more maintenance than Cloud Run and Cloud Spanner.

Is it? I keep quite some linux boxes running and they don't really require too much maintenance. Not to mention that when things do not work, I have complete visibility on everything. I doubt Cloud Run provides you with full visibility.

> It is not 0, it is just that if you manage cloud run and cloud spanner you don't manage all the other things you have to manage when you self host, and managing cloud run and spanner is really not a lot of effort, it is a lot less effort than managing a standalone database at least.

Managing a simple standalone database is not a lot of effort. For most cases, specially the ones that can get away with running production on a single box, you'll be ok with "sudo apt install postgresql". That's how much database management you'll have to do.

> Cloud run can run stock standard docker containers, you will have a bad time if your processes are not stateless though, and you will have the best time if you have a 12-factor app, but I would not count that as adapting to infrastructure.

That definitely counts as adapting to infrastructure. For example, if I want to use Cloud Run my container should start fairly quickly, if it doesn't I need an instance running all the time which increases costs.

I'm not saying Cloud Run/Spanner are bad. They'll have their use cases. But for simple deployments it's more complexity and more things to manage, and also more expensive. If doing "apt install postgres; git pull; systemctl start my-service" works and serves its purpose, why would I overcomplicate it with going to redundant systems, managed environments and complex distributed platforms? What do I stand to gain and at what cost?