It’s kinda curious that the HN sentiment in the comments has lately switched from “kubernetes is hell” to “kubernetes is quite useful and well worth it” To me, kubernetes is the kind of giant, complicated, hard to understand software that is ripe for HN disdain.
Kubernetes is a red flag signalling premature optimisation
161–170 of 558 posts
Re: Kubernetes is a red flag signalling premature optimisation
#162I've been doing all this with the help of docker-machine (now deprecated), and a collection of handy bash scripts. While this has worked, it's become more and more fragile and I need something better.
For years I've resisted Kubernetes because I keep hearing "premature optimisation", but at this point I'm not sure I have any other options. So before I dive head-long into Kuburnetes by what feels like necessity, how else could I possibly: - Push button provisioning of new hosts across a number of cloud providers - Monitoring so I can restart services under certain conditions. - Notification of failures etc. - Automatic / continuous deployment across all servers and regions.
Re: Kubernetes is a red flag signalling premature optimisation
#163It’s kinda curious that the HN sentiment in the comments has lately switched from “kubernetes is hell” to “kubernetes is quite useful and well worth it” To me, kubernetes is the kind of giant, complicated, hard to understand software that is ripe for HN disdain.
Re: Kubernetes is a red flag signalling premature optimisation
#164Earlier quoted context omitted.
That's sad, JavaScript was already not great for front-end and we now get it in backend and even the edge. Most job offers are for a mythical full-stack developper that'll master web design, CSS/HTML, front-end interactions and code, networking, back-end architectures, security,... You end-up with people who don't have time to get enough expertise and write and build clean stuff. Hacking poor JavaScript code everywhe…
I'm far from a full stack developer, but really how much code would actually be common across the front and and back end? I would have thought maybe some validation code, not sure how much else?
If you do server-side rendering with something like nextjs, then its quite a lot of code.
With trpc you can share types without going through an intermediary schema language (https://trpc.io/) although I think it would've still benefitted from separating the schema from the implementation (hope you don't forget the type specifier and import your backend into your frontend by accident)
For business logic as always the anwwer is "it depends". Does your app benefit from zero-latency business logic updates? Do you need rich "work offline" functionality combined with a rich "run automation while I'm away" for the same processes? etc.
Re: Kubernetes is a red flag signalling premature optimisation
#165Earlier quoted context omitted.
While concepts transfer over between languages the language is only 10% of it. The rest of it is the standard library, ecosystem, buildsystem and all kinds of intricacies you have to know about and what not which is specific to the language (ecosystem).
There is also an issue with conceptual leakage, most noticeably I've found with devs well versed in one language bending another language into behaving like the former.
I write elixir that smells like rails, but as the months move on, I change.
Re: Kubernetes is a red flag signalling premature optimisation
#166I've seen cases where we started off as simply as possible with no k8's. We built the initial product really quickly using a ton of managed services. Whilst it was great to get us going, once we hit "growth" things just didn't scale. (1) The cost of cloud was getting astronomical for us (and growing with each new deployment) and (2) it was totally inflexible (whether that be wanting to deploy in a different cloud, or extend the platform's featureset) because we didn't own the underlying service.
We ended up porting everything to k8's. That was a long & arduous process but it gave us total flexibility at significant cost savings. The benefits were great, but not everyone has access to the engineers/skillset needed to be successful with k8's.
That's why we built Plural.sh – it takes the hard work out of k8's deployments. I've seen people go from zero to a full production deployment of a datastack on k8's in just 2 weeks. It deploys in your cloud, and you own the underlying infra and conf so you have total control of it. And because we believe in being open, you can eject your stack out of plural if you don't like it and keep everything running.
Great post, and hope all is well with you!
Re: Kubernetes is a red flag signalling premature optimisation
#167Earlier quoted context omitted.
In my latest project I skipped the managed database and started using a postgres operator. So far the dev experience and performance is good.
The PostgreSQL operators are definitely usable, pretty much none of their limitations are going to hit you on an MVP. I think a lot of the FUD about "don't run databases on kubernetes" came from a time before we had statefulsets and persistent volumes. Which operator did you go with out of curiousity?
Re: Kubernetes is a red flag signalling premature optimisation
#168> Companies using Kubernetes for an application that is a web app.
There is probably a whole spectrum, with manually deploying code to shared hosting through SFTP being on one end and having full Infrastructure as Code running on Kubernetes with CRDs and other functionality on the other.
Personally, I think that you need to find the sweet spot:
- use containers, otherwise your app environments will rot over time, or at the least become inconsistent
- consider something like Ansible or another GitOps approach for managing the app config, consider the same for server seetup
- use some sort of container orchestration, whatever is easier and whatever you're more familiar with (Docker Swarm, Hashicorp Nomad, K3s)
- if you do go for Kubernetes (K3s or otherwise) make it as simple as possible, don't go for fancy service meshes or bunches of operators etc.
- document what you can: with Markdown or whatever for describing things that need to be known and done manually, code for the rest
This approach has allowed me to be able to add any server to a cluster easily, to distribute the load of all the app services, or even be able to treat many projects with different tech stacks the same way: just launch the container, configure resource limits, parameters, storage, networking/certificates and it's done. Things crash? Automatic restarts and liveness probes/health checks. Want the logs? Just look at the output or easily configure shipping them somewhere. Need to deliver new version to clients? Just push to Nexus/Artifactory/Harbor. And should your org have separate Ops people? Well, they won't really have to care about someone wanting to bump the JDK version or whatever, let the devs care about what's inside of the app themselves. Security concerns? Scan the whole container with Trivy? Clients have a different environment? They probably can run OCI containers somehow.Many of those advantages also apply to monolithic apps, like some garbage that may only run on JDK $OLD_VERSION or $OLD_DISTRO, but should still have its surface area for attacks limited as much as possible. If done right, developing apps like this will be a breeze without getting too overcomplicated. If done wrong, you will get nothing done and will have to suffer with figuring out how to get your Kubernetes cluster up and running properly, then will have problems with resource usage etc.
> More than one language for your application. For example, a backend in Golang, Ruby, PHP etc. and a Frontend Web App in React, Vue etc.
This is probably the most controversial argument: I will admit that using something like Ruby on Rails or Laravel, or any other server side rendering option will probably be a pretty decent and fast way to get up and running.
Yet, nowadays people like developing APIs so that other apps can be connected to those relatively easily, which is somewhat handled by most of these options as well, though developing a separate front end and dogfooding the API yourselves will perhaps be the better way to ensure that everything works as expected.
More so, in my personal experience trying to develop both the front end and back end as a single bundle can sometimes be a major pain in the behind, which gets infinitely worse when you do have some sort of a full web app that you try to embed and serve from the back end container/process/deployment as static files, because of issues with permissions, threads/performance etc. If anyone is curious, doing that with Spring (not Boot) was really annoying and kind of brittle, having to always deploy them together and having unreasonably long startup times aside.
So I can't help but to lean towards a split setup (in most cases):
- RESTful API for the back end, can be used by the front end or other integrations (I don't think that GraphQL is all that good, but it's also an option)
- some sort of a separate web app front end (Vue, React, Angular) that is served by Nginx/Caddy/Apache and may later be replaced/migrated over to another solution etc.
> Not using a cloud service to host your app. Examples are Heroku, Vercel, Netlify and Fly.io. Most product teams will have over-architected their solution if they have to have an ops or infra team.This probably depends on corporate policies or whatever people are comfortable with. Using those cloud services will result in vendor lock which, as Heroku showed, isn't always cost effective or even that good of an idea long term. Currently I pay around 260 Euros/year for my servers (5 VPSes) on which I run my containers, some people pay more than that per month on certain platforms. Because I settled on OCI containers as the common runtime format and the aforementioned approaches to orchestration, I can switch between whatever hosts that I want.
Previously, I've used DigitalOcean, Vultr, Scaleway, Hetzner, Contabo and Time4VPS (my current one) and can easily move, whenever a better offer comes along. Of course, for many startups with SV funding that doesn't matter much, but there is definitely a lot of merit to avoiding vendor lock and focusing on open source standards. Or, you know, you decide to work in an industry where the law book gets thrown at you and you have to do on-prem.
Re: Kubernetes is a red flag signalling premature optimisation
#169as someone who is currently tearing down a managed k8s cluster and migrating the services into SaaS I sympathise with this blog heavily
Re: Kubernetes is a red flag signalling premature optimisation
#170Earlier quoted context omitted.
Right.. But scale? I've seen places hire a dev that write all the OPS stuff and they scaled awesomely.. I mean if they had purchased 100servers full time on amazon, they would have spent a fraction of the cost to scale, but they could scale. In 5 years I think they've never once had to reach even near the 100servers. At the same time. I can scale heroku to 500 servers, and still be under the cost of one ops person. I…
> At the same time. I can scale heroku to 500 servers, and still be under the cost of one ops person. I can make that change and leave it there. I can do that all in under 30 seconds. Oh. And CICD is built in as a github hook. Even with blue-green deploys. And then Heroku shuts down. If you're building something that needs to scale up rapidly if it succeeds, k8s is worth thinking about. Either you don't succeed, in w…