EC2s are provisioned with Terraform or if you want to bypass that, click-ops. Nothing custom is needed. You can define a small shell script that is known as userdata that runs when the EC2 boots. That's standard and used everywhere that EC2s are used.
If you want more customization - again not necessary - you can build via Packer. The thing is, you have to do that anyways with Docker or whatever your container runtime happens to be. Docker images need patches as well if you install dependencies like openssl, libpql, etc.
ArgoCD - ArgoCD is great but it's something that has to be separate managed and comes with a database, UI layer, etc. and it's own set of permissions. The whole point of ArgoCD is to have deployments specifically around GitOps for container-based applications. Unnecessary if you don't have K8s.
Load balancers are simpler with ALBs and route directly to the auto-scaling groups. Traeffik is something that you need to manage. Also, K8s typically requires the use of external load balancers to work in the cloud as it is which means there's a layer of complexity that was introduced. Typically, K8s through the External Load Balancer Controller will provision the LB and attach targets. So it adds complexity. Traeffik isn't free, you are paying for the compute in your cluster which has the overhead of the control plane.
Let's Encrypt is another layer of complexity that has to be managed. AWS has ACM which can dynamically create and renew your certifications without you having to do anything.
K8s as a cluster needs to be managed and introduces significant breaking API changes across minor versions (1.23-1.24, etc.). Postgres managed as a stateful application also breaks some of the benefits of K8s in that it needs to be managed as a StatefulSet rather than as a ReplicaSet which means you likely need different deployment mechanisms to avoid destroying it accidentally.
Ironically, this architecture doesn't need any customizations at all. You can provision it with a few modules in Terraform or you can click ops the entire thing. K8s on the other hand, requires pretty constant attention. You've now introduced the management of the cluster, the building of the image, management of ArgoCD, Traeffik as well as external load balancers.
Basically:
- Route53 for domain management
- ACM for cert management
- EC2s for LBs
- ASGs to manage EC2s
- Small shell script within userdata to configure the host vm
- RDS if you need a database
Avoiding the operational and actual costs of K8s.
I think that there are compelling reasons to use K8s:
- Lots of smaller services that need to interface with each other
- Product is deployed to multiple clouds or in a hybrid setting
- Infrastructure team has deep expertise in K8s