Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do I Need Kubernetes?

mbird.biz

Post reply on HN
Print view View original

41–50 of 180 posts

Re: Do I Need Kubernetes?

#41

Post by cat199 »

mrweasel wrote: post #3

I only have one tiny detail: Bursty traffic means that your cluster needs to be able to deal with the peaks. If you running an on prem Kubernetes cluster, then there is no savings, unless you can use the capacity for something else during non peak periods. The scaling, and potential savings is a cloud feature, not a feature of Kubernetes.

> no savings

This only relates to hardware cost - tooling impact can provide cost improvements if it enhances productivity (not always clear for k8s case)

Re: Do I Need Kubernetes?

#42

Post by abatilo »

gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

There are a lot of projects these days that aim to use Kubernetes as a docker-compose environment. I personally use http://skaffold.dev/ with either a local Kubernetes cluster built with https://github.com/rancher/k3d or https://github.com/kubernetes-sigs/kind. I think there's a very easy argument to be made that says that running K8s locally is overkill, but what I will say is that if you run your applications locally in K8s, that's one step closer to having your local environment mirror the production environment. Couple that with things like running https://github.com/localstack/localstack locally and you get even closer.

Re: Do I Need Kubernetes?

#43

Post by solatic »

spost wrote: post #23

I work for a startup whose product is small (half a dozen servers, if relatively beefy ones) clusters that will be run on-prem by customers, at least sometimes in a low-to-no-touch capacity. Most of our application components are micro-ish services that are run on all hosts in the cluster for either extra capacity or fault tolerance. We currently run everything on mesos/marathon, but are looking to switch away from i…

> But I never really see k8s being talked about in that context of “physical hardware that’s on prem, but not on MY prem.” Is there a reason for that? If we go with k8s is it going to bite us? Does anyone have experience with something like that they could share?

Kubernetes provides a leaky abstraction above the underlying hardware - the storage and networking are going to be different depending on who is maintaining the Kubernetes cluster. Kubernetes's strength is that it acknowledges the leakiness of the abstraction and makes it explicit. If your customer uses a specific networking and storage provider, Kubernetes makes it easier for you to say (or not) that you have certified your product for those networking and storage providers, and here's what the manifests look like, because there's a standard way of configuring the application to work with that networking (CNI, which powers the standard Service as well as maybe NetworkPolicies) and storage (CSI, specifically StorageClass) provider.

If you just provide Docker images, or VM appliances, then Murphy promises you that you're going to get frustrated support calls from customers saying "your application is slow and we don't understand why." Good luck then.

Re: Do I Need Kubernetes?

#44

Post by solatic »

irontinkerer wrote: post #7

> To make a cluster useful for the average workload a menagerie of add-ons will be required. Some of them almost everyone uses, others are somewhat niche. This is the concern I have with k8s. All this complexity introduces operational and security concerns, while adding more work to do before you can just deploy business value (compared to launching on standard auto-scaling cloud instances)

As a rule of thumb, the quality of engineering of the core Kubernetes distribution is rock solid and incredible, but anything that's not is in varying stages of maturity.

How much Kubernetes-adjacent code you actually need to adopt (and therefore, how much risk you take on) depends from project to project and organization to organization.

Re: Do I Need Kubernetes?

#45

Post by weedpeg »

gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

Agreed with all mentioned upsides of dockerzing apps. One disadvantage though is a lot of additional work is required for APM/logs/infra monitoring. Even in 2020, newrelic/datadog/others documentation seems to be written non-containerised setup.

Re: Do I Need Kubernetes?

#46

Post by spanhandler »

gravypod wrote: post #27
JamesSwift wrote: post #21

Earlier quoted context omitted.

I prefer to take it one abstraction further and setup a makefile where the default `make` brings up the app. This makes it so you can rework the runtime as you see fit and nothing changes. It also doesn't require you to jump into docker immediately, and gives a good extension point for various 'helper script' workflows that you will always have.

Makefiles are nice in theory but don't scale well to team members who don't know how to use make. If your team is full of experienced C/C++ devs then this will probably work great for you. If you are working with a bunch of android/javascript/etc developers who've never seen make, and who haven't exercised the skillset of reading through 3-decade-old email threads to find answers, you'll find that the only thing you'…

Yeah but if you use bare tools then it’s hard to change them and the interface may differ project to project. If you let the JS developers pick your universal build tool then they’ll want to change it annually because now the old one they loved is “terrible” and some other thing is great and is definitely the way forward. Plus then you’ll need node on every machine that needs to run the build, and probably you’ll want to lock the version and not use the one in LTS OS releases because those are too old so now all your poor non-JS developers are installing NVM and Node and a pile of other crap.

Or you can just make everyone wrap their build tools in make.

Re: Do I Need Kubernetes?

#47

Post by halbritt »

gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

This facilitates a benefit that I think has been described as "self-contained systems". There's one website on the topic, that imho is too narrowly focused and seeks to distinguish itself from microservices incorrectly, I think.

That said, a self-contained system in the context of a container orchestration framework, docker-compose, kubernetes, etc, facilitates an number of benefits, not the least of which is the ease of scaling engineering staff.

For example, in typical organizations that claim to have modern development practices, there are static dev, test, and staging environments for a given service. Obviously these are subject to config skew, etc. and must be time-shared between folks. When it's a simple task to create an environment per engineer for each purpose or any other, that can accelerate development tremendously.

There are other benefits around resilience in production, but that's another topic entirely.

Re: Do I Need Kubernetes?

#49

Post by halbritt »

tstrimple wrote: post #24
mrweasel wrote: post #3

I only have one tiny detail: Bursty traffic means that your cluster needs to be able to deal with the peaks. If you running an on prem Kubernetes cluster, then there is no savings, unless you can use the capacity for something else during non peak periods. The scaling, and potential savings is a cloud feature, not a feature of Kubernetes.

It also enables you to achieve app density. Many of the companies I've been working with lately have large batch processes on nightly / weekly / monthly basis. For some reason, each job previously was setup on on-prem hardware dedicated per workload. Using Kubernetes and scheduling jobs to manage capacity has enabled us to reduce the number of on-prem servers substantially as part of the migration plan for the cloud.

Precisely.

In my experience hosting services in VMs on-prem I was able to achieve roughly 30% efficiency across 3k instances and hundreds of nodes. In my experience hosting microservices on k8s, I was able to achieve 80% efficiency across hundreds of nodes.

Both were the result of a great deal of work to optimize efficiency. In this case, I use the word "efficiency" to refer to a blend of CPU and memory utilization.

Re: Do I Need Kubernetes?

#50

Post by thiht »

gravypod wrote: post #27
JamesSwift wrote: post #21

Earlier quoted context omitted.

I prefer to take it one abstraction further and setup a makefile where the default `make` brings up the app. This makes it so you can rework the runtime as you see fit and nothing changes. It also doesn't require you to jump into docker immediately, and gives a good extension point for various 'helper script' workflows that you will always have.

Makefiles are nice in theory but don't scale well to team members who don't know how to use make. If your team is full of experienced C/C++ devs then this will probably work great for you. If you are working with a bunch of android/javascript/etc developers who've never seen make, and who haven't exercised the skillset of reading through 3-decade-old email threads to find answers, you'll find that the only thing you'…

> Makefiles are nice in theory but don't scale well to team members who don't know how to use make

Just learn it then? Running make commands is trivial. Writing basic make targets is trivial too.

Post reply on HN