Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do I Need Kubernetes?

mbird.biz

Post reply on HN
Print view View original

91–100 of 180 posts

Re: Do I Need Kubernetes?

#91

Post by overgard »

throwaway894345 wrote: post #73
overgard wrote: post #65

I think one of the underrated parts of using something like Kubernetes early (or even w/ simpler orchestrators like swarm or rancher), is that it encourages (and sometimes enforces) architecture best practices from the start. IE, you won't be storing state locally, you'll be able to handle servers being randomly killed, you'll already have horizontal scaling, etc. In my experience the hard part of migrating to contai…

I’m not avidly opposed to k8s by any means, but you can get these same properties from any of a variety of easier-to-use schedulers such as Fargate, Heroku, or even EC2 autoscaling groups. Of course, there are probably Kubernetes distributions that lower the threshold of using Kubernetes (and if there aren’t, there really should be) by providing solutions for logging, monitoring, certificate management, Functions (a…

So, I think the downside of those systems is that they tend to be specific to a provider, so you get some lock in, and not all of those things are easy/possible to run locally, whereas minikube or docker swarm can replicate a cluster locally, which I think is really important. Otherwise yeah, those systems can be used to accomplish the same goals.

One thing that I think is non-obvious: there are easy ways to set up a kubernetes cluster, it's just that there's so many options out there that the easier solutions tend to get lost in the noise. If I had to manage kubernetes nodes manually I'd probably never do it, but setting up something like AKS/EKS/GKS -- they all have their own issues, but they're pretty easy to get started with.

Re: Do I Need Kubernetes?

#92

Post by watermelon0 »

abatilo wrote: post #42
gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

There are a lot of projects these days that aim to use Kubernetes as a docker-compose environment. I personally use http://skaffold.dev/ with either a local Kubernetes cluster built with https://github.com/rancher/k3d or https://github.com/kubernetes-sigs/kind . I think there's a very easy argument to be made that says that running K8s locally is overkill, but what I will say is that if you run your applications loca…

Neither k3d (k3s) nor kind support building of images since they use containerd directly, which [building] seems to be a requirement for development environment.

This means that you need to build your app using Docker, and then import it to k3d/kind environment, thus needing local registry (not enabled by default), which means that images are duplicated.

Re: Do I Need Kubernetes?

#93

Post by secondcoming »

overgard wrote: post #65

I think one of the underrated parts of using something like Kubernetes early (or even w/ simpler orchestrators like swarm or rancher), is that it encourages (and sometimes enforces) architecture best practices from the start. IE, you won't be storing state locally, you'll be able to handle servers being randomly killed, you'll already have horizontal scaling, etc. In my experience the hard part of migrating to contai…

> I good reason not to use kubernetes though is if you know your app is probably never going to scale

Suggesting that you can only scale with Kubernetes just isn't true. Our tech stack originally ran on AWS in multi-regions. Then we moved to GCP. The architecture was the same for both:

- public facing Load Balancer

- instance groups attached to the LB

- add health checks so unresponsive instances get killed

- add scaling rules based on whatever metric you want (AWS's offering was far better than GCP's)

- choose your instance type(s).

And for the record, we handle ~15 billion HTTP requests per day (traffic is seasonal). Our busiest data centre is us-east, which has a max of 50 instances (16 vCPU, 31GB RAM), and we rarely need them all. You can also mix spot and non-spot instances too, so we know we have a least one or two machines that GCP won't kill.

Rolling out updates involves building a debian package, building a new VM image, and then clicking a 'deploy' button (GCP makes this easier than AWS).

Re: Do I Need Kubernetes?

#94

Post by escardin »

spost wrote: post #23

I work for a startup whose product is small (half a dozen servers, if relatively beefy ones) clusters that will be run on-prem by customers, at least sometimes in a low-to-no-touch capacity. Most of our application components are micro-ish services that are run on all hosts in the cluster for either extra capacity or fault tolerance. We currently run everything on mesos/marathon, but are looking to switch away from i…

I did an on-prem k8s deployment at my last place. It is definitely challenging compared to EKS and GKE, but the difficultly is not in base k8s.

Following the kubeadm getting started guide on the kubernetes.io site can get you an 'ha', 'production ready' going in a couple hours. Most of it is pretty mechanical, and only needs a couple key decisions, mainly your networking plugin. Generally the most popular ones have instructions as part of the getting started guide, making the process straight forward.

Where it quickly becomes difficult is after this step. You have a cluster ready to serve workloads, but it has no storage, no ingress/external load balancer.

Storage can be as simple as NFS volumes (you don't even need a provider for this, but you should use one anyway). Rook/Ceph will work, but now you've just taken on two complex technologies instead of one.

Without an external load balancer of some sort, you will have trouble getting traffic into your cluster, and it likely won't be actually HA. You can use MetalLB for this, or appliances. If you're just starting out though, you can totally get away with setting up CNAME aliases in DNS to your nodes in a round robin type fashion. It won't be HA, but it will work, and is simple and straight forward.

Ingress is pretty easy to setup for the most part. Usually just applying an available manifest with a tweak or two. If you go the CNAME route, you will need an ingress setup so you can serve http/https on standard ports without too many issues.

If you do all these things, then you have a real deal cluster. Things like ingresses are recommended even if you're running in the cloud, so you may find that you're not all that far off from what you might find there.

Overall, the biggest trouble is all the choices you need to make. If you're starting out, maybe read up on two or three of the most popular choices for each step, and then just pick one. Anything that exists entirely within the cluster can usually be expressed purely as source controlled manifests, and kubeadm deployments can be simple shell scripts if you don't make them do everything (i.e. only support one container driver, not all of them).

One major caveat; If you screw up your network layer, you basically have to start over. This isn't strictly true, but it's the one where you are often better off starting over when you need to make fundamental changes to your network setup (like podCIDR and serviceCIDR or your network plugin). Pretty much everything else can be made to work with multiple setups at once, or you just need to delete and redeploy that component.

Re: Do I Need Kubernetes?

#95

Post by watermelon0 »

peterwwillis wrote: post #13

Here's a quick reference: 1) Are you on AWS? Then you don't need Kubernetes. Use Fargate. 2) Are you on Google Cloud? Then you don't need Kubernetes. Use Cloud Run. 3) Are you on Azure? Then you don't need Kubernetes. Use Azure Container Instances. 4) Are you on a PaaS like Heroku? Then you don't need Kubernetes. 5) Are you on a random VPC provider / bare metal machines? You could probably still do without Kubernetes…

So you are replacing one orchestration (Kubernetes) with another (e.g. Fargate + ECS on AWS)?

Re: Do I Need Kubernetes?

#96

Post by freehunter »

threeseed wrote: post #85
throwaway894345 wrote: post #83

Earlier quoted context omitted.

Vendor lock-in concerns are overblown. Unless there’s a real chance you’ll need to pick up and move, don’t worry about it. Your savings by not building/operating everything yourself will dwarf other costs (unless your business has huge scale and you have a world class internal cloud capability which you probably don’t and if you do, you can probably just negotiate a better deal from a cloud provider a la Netflix and…

Nobody is saying you have to build everything yourself. But you can just use cloud providers for their hardware and not needlessly tie yourself to their software. For example you can use a managed K8s solution like EKS but then have all of your monitoring, logging, databases etc all be self-hosted. And it's not just about cost but also about being able to take advantage of other cloud provider's unique strengths or b…

> being able to take advantage of other cloud provider's unique strengths

If you’re limiting yourself to using only easily portable services, you’re not going to be taking advantage of any cloud provider’s unique strengths anyway. So you might as well just pick one.

Re: Do I Need Kubernetes?

#99

Post by pc86 »

gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

I would love to see an example of how to set something like this up if you've got any links in mind.

We've got multiple apps deployed various ways, everything from pseudo-automated CI/CD (auto build but manual deploy) to fully manual deployment directly out of our IDE. Nothing set up within Docker containers yet except for a few proofs of concept.

Re: Do I Need Kubernetes?

#100

Post by drran »

thom wrote: post #9
gravypod wrote: post #6

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and…

Can I ask people who _don't_ use Kubernetes and maybe have architectures built on proprietary cloud services: how do you manage this?

Everything is packages in RPM, so simple dnf install on bare metal or in container is enough.
Post reply on HN