Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I recommended ECS instead of Kubernetes to my latest customer

leanercloud.beehiiv.com

Post reply on HN
Print view View original

11–20 of 126 posts

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#11

Post by KaiserPro »

waffletower wrote: post #9

The management of infrastructure via Terraform has a hidden engineering cost that should also be considered. Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes, despite its own complexity, given the immature, inconsistent and undeniably awkward qualities of the Terraform toolchain. Engineering time is expensive -- the morass of Terraform can easily quadruple engineering effort…

> Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes, despite its own complexity.

Citation needed.

K8s has a whole bunch of footguns that people who don't want to manage infra can easily blunder into.

Terraform and ecs is not immature, and its fairly simple to maintain especially if they are just pushing updates without significant infra changes. (ie bumping the container version)

> Engineering time is expensive

which is why ECS is probably better, because its good enough for running a few containers that talk to a load balancer.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#12

Post by x86x87 »

whatever1 wrote: post #3
itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

Exactly. Worry about scaling when scaling is in the horizon

No no no. We want to be like Google. Web Scale. Big big data. Huuge

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#13

Post by sigstoat »

itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

i had a client who was burning… $10k? maybe $20k per month largely on nodes for EKS when they had no paying customers and ~zero load. (they had fully “production” sized clusters in all of their environments, and they had a slew of weird not-quite-prod environments.)

they also had some rabbitmq-on-k8s system going that fell over during small tests because they couldn’t get k8s to actually scale it. (which then convinced them they needed k8s, and bigger nodes)

sigh

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#14

Post by 0xEFF »

itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

A single EC2 instance is an equally bad trade-off on the opposite side of the spectrum from over architected SQS, SNS, etc…

The ideal trade off is a single Kubernetes cluster with as much in the cluster as makes sense for the team and stage of the project. As you say, toss the app on a single node to start, but the control plane is tremendously valuable from on the onset of most projects.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#15

Post by mywittyname »

waffletower wrote: post #10
itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

And in that case, why ec2, why not a more affordable provider?

There's a lot of expertise in AWS-land.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#16

Post by taeric »

itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

It is rather amusing how over engineered most seed projects have a tendency to be.

I do think ddb and lambda hit a sweet spot for costs on ramping up. The rest, though, really struggle.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#17

Post by ljm »

itsmemattchung wrote: post #2

> When looking at the cloud resources, we noticed many On-Demand EC2 instances with relatively low CPU utilization, which can be expected considering they don't have customers yet. As a software consultant myself, I'd probably stop the conversation right there and ask why they are building such a robust distributed system — SQS, SNS, etc — without any customers. Still want to be deployed in AWS? Toss the damn app on…

I’ve been exploring this lately because, honestly, the cloud is total overkill for small startups and hobby projects.

Kubernetes has its value even for small scale workloads like that, but it’s still a few steps more than, say, running a Capistrano script to push your code to a small Linux box with a database on a second one.

You’ll get really far on minimal resources these days, especially with cheaper ARM boxes that offer far more bang for your buck. Paying 1k+ a month to AWS/GCP/Azure is total insanity when you’re not even averaging a single active user a day.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#18

Post by TurningCanadian »

danpalmer wrote: post #7

More accurately: "Given using AWS as a requirement, I recommended ECS instead of K8s". It's not really surprising that AWS's K8S setup isn't great, and their own implementation ties in more closely with other services they offer. It's lock-in. AWS provides just enough K8S to tick the box on a spec sheet, but have little incentive to go beyond that.

The nice thing about a standard like K8S is how there are other clients to choose from.

You can do everything from the CLI with kubectl of course, but there are also a bunch of apps that will work with any K8S cluster:

https://medium.com/dictcp/kubernetes-gui-clients-in-2020-kub...

It's very nice to have a consistent interface across multiple cloud providers.

Re: Why I recommended ECS instead of Kubernetes to my latest customer

#19

Post by twalla »

waffletower wrote: post #9

The management of infrastructure via Terraform has a hidden engineering cost that should also be considered. Engineers can much more easily maintain, learn and introspect infrastructure via Kubernetes, despite its own complexity, given the immature, inconsistent and undeniably awkward qualities of the Terraform toolchain. Engineering time is expensive -- the morass of Terraform can easily quadruple engineering effort…

As a something of a k8s maximalist, I kind of disagree here. I think, especially for early stage and smaller teams, TF ends up being "closer to the metal" in the sense that there are fewer concepts and abstractions that need to be understood before an engineer can build a model of the resources they want, how they are grouped and how state reconciliation works. With k8s you're really just trading out crappy third party modules and providers for crappy operators and controllers.

I do think that as organizations grow, the ability for components to be defined in smaller units without being enmeshed in a big-ass tf dependency graph is a big draw of the controller model. The flipside is this comes with accepting the operational overhead of k8s plus the attendant controllers/operators you're running and hiring/staffing accordingly. There are ways you can structure your terraform that avoids creating the tight coupling some folks don't like where you have to literally define the entire universe to change a machine image. Not to mention, there do exist tools that allow you to inspect and visualize tf state.

Post reply on HN