> managed K8s (GKE or EKS) you can absolutely get a pretty well set up infra very quickly And then tear your hair out when something doesn't work for some reason and root causing it requires learning a stupid number of layers. k8s is easy until it goes wrong.
Isn't this the same for every software? How is debugging issues with Linux, NGINX, any complex framework any easier?
Well, for one thing, all logs are in /var/log.
If it doesn’t fail just right in kubernetes there might be no logs at all.
I’m thinking on particular about trying to mount filesystems into pods.
I'm also on a 2-person project on DigitalOcean k8s, also very happy. K8s is kind of messy compared to Heroku, which I don't love, but is also way more powerful and can be more secure. I don't know what I'd use instead of it, exactly as you said. Also, we run a VPC-only K3s node for some simple internal tools that works great as well.
> Also, we run a VPC-only K3s node for some simple internal tools that works great as well. We do exactly the same thing! We have a one-node k8s for all these dev things that just works. Everything is containerized for local dev anyway so moving it to k8s was just writing the deployment manifest. On heroku, all of these would be separate dynos (or one glued-together dyno that does everything). On a self-hosted VM we'…
I feel like I’m witnessing two co-founding colleagues - who sit by each other day in and day out - discover the other’s persona on HackerNews.
Sure, maybe you two (@dvcrn and @arcticfox) don’t work together and don’t know each other, but it’s definitely more entertaining imagining the scenario above.
I work for a mid size company with 30-40 engineers managing 20-30 very diverse apps in terms of scale requirements and architectural complexity. It took our devops team (4-5 people) probably 18 months to learn and fully migrate all our apps to Kubernetes. The upfront cost was massive, but nowadays the app teams own their deployments, configurations, SLA's, monitors, and cost metrics. Introducing Kubernetes into our o…
One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own large buffer to handle bursts of traffic.
EKS is a feature parity product. The pricing makes that painfully obvious.
We've been using EKS since it went GA. We haven't had a single control plane outage that I am aware of.
I'm sure it works fine if you can get it running (documentation didn't work when I played with it). I'm referring more to the $200/month it was per control plane.
That to me is a product they offer because someone else is offering it, but they don't want you to actually use it.
The 'needing a team' aspect of Kubernetes sounds remarkably similar to conversations I had like 8 years ago when Openstack was the new hotness. We went with ECS and have been happy with it. It plays well with all of AWS's other products and features. For the few things we have to run On-Prem we use Docker Swarm in single node mode and it works well (albeit missing a few features like crons from Kubernetes).
AWS Kubernetes (EKS) with Fargate is just as simple as ECS. Far simpler if you can include the fact that with Kubernetes you can install new applications as simply as "helm install prometheus".
Are they still charging something ridiculous for control planes? When I looked at it they were like $200/month.
I wasn't aware they were offering fargate on EKS now. They weren't when I looked at it last.
> Also, we run a VPC-only K3s node for some simple internal tools that works great as well. We do exactly the same thing! We have a one-node k8s for all these dev things that just works. Everything is containerized for local dev anyway so moving it to k8s was just writing the deployment manifest. On heroku, all of these would be separate dynos (or one glued-together dyno that does everything). On a self-hosted VM we'…
I feel like I’m witnessing two co-founding colleagues - who sit by each other day in and day out - discover the other’s persona on HackerNews. Sure, maybe you two (@dvcrn and @arcticfox) don’t work together and don’t know each other, but it’s definitely more entertaining imagining the scenario above.
That's fair; all of this is heavily influenced by your operating environment. If you can run GKE or if you have an ops team to deal with that stuff, then yeah k8s is great. Unfortunately, I'm part of the ops team, and our company is too small to have a dedicated k8s team and too low-budget to (likely) do well with a managed service (we get absurd value per money out of bare metal servers, which is very much a tradeof…
> we get absurd value per money out of bare metal servers, which is very much a tradeoff that is sometimes painful What do you find most painful about your use of bare-metal servers? The thing that I like most about a hyperscale cloud provider is the level of redundancy, including even multiple data centers per region, and their built-in health checks and recovery (e.g. through auto-scaling groups) based on that redu…
I don't know that it's a single big thing; it is indeed many small things that we have to manage ourselves. We backup the databases with our own scripts, we failover manually (I miss RDS), we use an overlay network because no VPC, and deployments involve ansible running docker-compose. There's basically no elasticity; we provision massive bare metal servers with fixed memory and disk installed. But, it is dirt cheap, so we manage, and all the pieces are small and easy enough to use.
> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…
I work on a 2-person project and decided to go with kubernetes (through digitalocean) for the cluster. I am managing everything with terraform and I don't have any big problems. I like that I can write everything as terraform manifests, have it diffed on git push and applied to prod if I want to. Sure it had a learning curve but now I just describe my deployments and k8s does the rest, which then reflects back on dig…
I'm on a 0.05 person project, using K8S through AKS
> We would need to build/staff a full-time Compute team This actually was a very real problem at my current job. The data pipeline was migrated to k8s and I was one of the engineers that worked to do that. Unfortunately, neither myself (nor the other data engineer) was a Kubernetes guy, so we kept running into dev-ops walls while also trying to build features and maintain the current codebase. It was a nightmare. If…
Did you evaluate Cloud Run/Fargate for deploying the same Docker images without k8s?