Distributed systems are cool, but people I think got carried away with complexity. Hardest part here is active-active replication, but this is needed for very few who need true 100% availability and big scaling. For majority of people active-backup setups are perfectly fine. You can also run frontend-backend mode where frontend takes care of caching, passing write to backend nodes. I run couple of systems like this and lives good. Failure of frontend node is absolutly no brainer. Failure of backend node with active-backup is also easy. You just lose last not commited and not synced transactions.
Kubernetes and back – Why I don't run distributed systems
21–30 of 43 posts
Re: Kubernetes and back – Why I don't run distributed systems
#22These posts come up so frequently, people keep falling for buzzwords they don't understand and then start blaming the tech. Distributed systems are all cool and fun but use them as a last option when you have exhausted all other choices. > I took a moment to calculate the cost of hosting my app on the K8s offerings of a Hyperscaler like AWS, Google or Azure just to figure out that this won’t be an option for me. Clea…
"You are holding it wrong", huh?
From the homepage https://kubernetes.io/:
"Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications."
Do you see "not recommended for smaller-scale applications" anywhere? Including on the entire home page? Looking for "small", "big" and "large" also yields nothing.
Re: Kubernetes and back – Why I don't run distributed systems
#23These posts come up so frequently, people keep falling for buzzwords they don't understand and then start blaming the tech. Distributed systems are all cool and fun but use them as a last option when you have exhausted all other choices. > I took a moment to calculate the cost of hosting my app on the K8s offerings of a Hyperscaler like AWS, Google or Azure just to figure out that this won’t be an option for me. Clea…
> use them as a last option when you have exhausted all other choices.
if one wants to run service with high availability (no downtime if server died), what are other choices then?..
Re: Kubernetes and back – Why I don't run distributed systems
#24Earlier quoted context omitted.
> A dedicated server from OVH running some processes under lxc and systemd can achieve 100% yearly uptime. Your hobby project isn’t Google. 73.6% of All Statistics Are Made up https://www.datacenterdynamics.com/en/opinions/ovhclouds-dat...
18:37:01 up 1618 days, 6:40, 6 users, load average: 0.00, 0.00, 0.00 Just go away and burn ;)
survivalship bias..
Re: Kubernetes and back – Why I don't run distributed systems
#25> All I’m saying is: Choose your weapons carefully.
That's the right moral of the story to take away.
> Today, I’m back with a single machine that handles the peak 500 concurrent users
And this is what tells me k8s is probably not the right tool for the job. Something that runs on a single machine should absolutely run on a single machine... Unless if there's a pre-existing, funded and operated cluster you can tap into.
No abstraction comes for free, as the author mentioned you need to figure routing, backup, etc. You are paying a premium for the fact that you want to make that somewhat generic enough to fit with multiple use cases. The abstraction doesn't materialize for the operator of the cluster. It's there for the customers of that cluster - the devs who can now avoid to configure whatever the cluster is already handling.
Re: Kubernetes and back – Why I don't run distributed systems
#26Earlier quoted context omitted.
18:37:01 up 1618 days, 6:40, 6 users, load average: 0.00, 0.00, 0.00 Just go away and burn ;)
survivalship bias..
the original comment was about the possibility of achieving 100% uptime, not that it's an expected outcome
> A dedicated server from OVH running some processes under lxc and systemd can achieve 100% yearly uptime.
Re: Kubernetes and back – Why I don't run distributed systems
#27We ran our project on Google's K8s for $76 a month for quite a while. They provide one free Master. There is no way, EVER, that I would host K8s myself for the size of the projects I usually work on. But having a panic in one of my services at 1am, and K8s auto-restarting the pod is amazing. Namespaces even let us run three separate environments for that same $76. We have a couple thousand users per day who are in the app all day, so not a large project. We use other services like GCP Cloud Storage, Pub/Sub, PostgreSQL, etc. that require next to no setup. We are up to 3 nodes in our small cluster and still costing us less than $350 a month.
K8s is repetitive. You create one YAML for a service and the rest are similar. Rolling releases are a simple script. As small as our company and product is, I think K8s has been one of our best decisions.
Some things like MongoDB can quickly triple your costs though. We were able to pick technologies that kept the cost really low and stack maintainable.
Re: Kubernetes and back – Why I don't run distributed systems
#28These posts come up so frequently, people keep falling for buzzwords they don't understand and then start blaming the tech. Distributed systems are all cool and fun but use them as a last option when you have exhausted all other choices. > I took a moment to calculate the cost of hosting my app on the K8s offerings of a Hyperscaler like AWS, Google or Azure just to figure out that this won’t be an option for me. Clea…
> use them as a last option when you have exhausted all other choices. if one wants to run service with high availability (no downtime if server died), what are other choices then?..
Load balancers in front of a Proxmox cluster + Ceph storage (managed inside Proxmox) + federated/replicated databases (MySQL/MariaDB/Postgres etc)
Re: Kubernetes and back – Why I don't run distributed systems
#29These posts come up so frequently, people keep falling for buzzwords they don't understand and then start blaming the tech. Distributed systems are all cool and fun but use them as a last option when you have exhausted all other choices. > I took a moment to calculate the cost of hosting my app on the K8s offerings of a Hyperscaler like AWS, Google or Azure just to figure out that this won’t be an option for me. Clea…
> These posts come up so frequently, people keep falling for buzzwords they don't understand and then start blaming the tech. Distributed systems are all cool and fun but use them as a last option when you have exhausted all other choices.
I've been in the SaaS application business since the early 2000's. When I first started, the big infra question was running on in-house infra or data-center infra. If it was the latter, then the follow-up question was which hosting provider. Regardless, as an Engineer, you were likely touching something that looked and felt like a variation of the box sitting under your desk.
Eventually we got cloud providers, serverless, containers, etc. Growing through the progression of technologies, they all seemed like a natural evolutiion of things to solve certain problems with other approaches.
For newer engineers starting out, they may only have seen these latter things, and at the scales and circumstances that make them a good choice. As such, it may not be that they're the new and shiny thing, but that they're the _only_ thing people have seen.
Re: Kubernetes and back – Why I don't run distributed systems
#30Earlier quoted context omitted.
> A dedicated server from OVH running some processes under lxc and systemd can achieve 100% yearly uptime. Your hobby project isn’t Google. 73.6% of All Statistics Are Made up https://www.datacenterdynamics.com/en/opinions/ovhclouds-dat...
18:37:01 up 1618 days, 6:40, 6 users, load average: 0.00, 0.00, 0.00 Just go away and burn ;)
Impressive uptime.
Sidenote: I used to take pride with these kinds of uptime. I now see it as a significant risk. If I take over a machine with an uptime of more than a month, I assume that the machine cannot survive a reboot. Which means I spend a lot of time trying to figure out how to deploy that server with all the data it needs. These days there’s likely deployment systems in place that prevent some of the horrors I’ve seen, but there was a time in the not to distant past when you had to assume someone ssh’d into a box at least once a quarter to muck with something. If they didn’t reboot, there was no guarantee the changes they made would survive.