31–40 of 63 posts
Would love to explore this in Python. But would it be correct to assume single process service would not be as performant due to GIL?
Wholly agree. Too often we think about scale far too early. I've seen very simple services get bogged down in needing to be "scalable" so they're built so they can be spun up or torn down easily. Then a load balancer is needed. Then an orchestration layer is needed so let's add Kubernetes. Then a shared state cache is needed so let's deploy Redis. Then we need some sort of networking layer so let's add a VPC. That's…
Also, I think people vastly overestimate how much uptime their application really needs and vastly underestimate how reliable a single VPS can be. I currently have VPSes running on both lowend and big cloud providers that have been running for years with no downtime except when it restarts for updates.
This sounds a little like saying "all of North America except the U.S."
I don't think people are worried about random breakdowns on a single VPS, but scheduled updates are still downtime, and downtime causes revenue loss regardless of why it happened.
Any time a service is important enough I ask for two servers and a load balancer specifically to handle deployments and upgrade windows transparently. But! I agree services are usually less important than people think.
5 minutes sounds reasonable
Wholly agree. Too often we think about scale far too early. I've seen very simple services get bogged down in needing to be "scalable" so they're built so they can be spun up or torn down easily. Then a load balancer is needed. Then an orchestration layer is needed so let's add Kubernetes. Then a shared state cache is needed so let's deploy Redis. Then we need some sort of networking layer so let's add a VPC. That's…
Even moreso, I could introspect the entire application state, including providing myself a shell and modify application state within the application. Keeping a blocklist inside a simple array- no problem! And being able to run a shell inside the same process meant I could inspect and even modify the array while the application ran.
That made it incredibly pleasant to use and run.
On the flip side, upgrades can be very challenging.
In a modern web application it's standard practice to run (at least) two instances of the application at once and use the load balancer to test both, or to drain jobs from one to the other. This is relatively easy if the applications are stateless.
Once the application holds all the state in memory, there's a real challenge. That array that seemed so clever- you'll need to serialize it so it can be reloaded at initialization time. Keeping all the session identifiers in memory- be ready to dump that.
Worse, if the application is not designed to share this state with another application, you're now in some trouble. This is fine if you're running a small site with a few users who can accept some downtime, but if you're running a serious service, you'd like to have some kind of upgrade path other than shutting the application down and starting it back up again.
"I'll just share state with other application servers", you might think, and then use something like ZeroMQ to transmit state", but once you think about sharing state between application servers, you realize you'd probably be better off using a tool like Redis, and you're right back where we started.
Then why not go to the extreme and just build a static version and serve that? Why do you need any dynamic content at all? Then you won't need shared state, then you won't need a database, then you won't need 90% of the stuff that dynamically driven pages need. Be the l33t h4x0r you are and just go static, save the complexity for your build process, because that shows the true hacker spirit. Hell, you may even be abl…
And your surface-level scans indicate a lot of specialized deep-thinking about some specific tools. Sure, but you'll also find some good generalizations that arose from the depth and breadth of experience. He knows Linux like the back of his hand, and he's been using Debian and Ubuntu, and Fedora, so perhaps we can derive some takeaways from those? And thoughts on ZFS and anti-spam email hosting, those are good too.
cks is the guy who influenced me to run Byron's rc shell, and also to install and run MH as my mail reader in 1993, and he also singlehandedly convinced me to install Ubuntu in 2006, which I maintained through multiple computers and upgrades through 2020. I cannot say that many, if any, of his blog posts were directly helpful to me, except for his opinions on PC hardware such as PCIe, parity RAM, and the like. But his wisdom is truly inspiring, as is his ability to stay with one employer for 100% of his career, doing more or less the same sysadmin things as he did in 1995.
Rust. Axum. Single compiled binary. Even html/js/css is baked into it via RustEmbed. Sqlite + litestream to S2. Cloudflare in front.
Works extremely well.
All that's old is new again. More than a decade ago, I inherited a Node application and was able to use many of these exact techniques inside the application. Even moreso, I could introspect the entire application state, including providing myself a shell and modify application state within the application. Keeping a blocklist inside a simple array- no problem! And being able to run a shell inside the same process me…
Would love to explore this in Python. But would it be correct to assume single process service would not be as performant due to GIL?
I’ve been running essential production systems with pythons uvicorn and 1 worker reliably, mainly because I get to use global state and the performance is more than fine. It’s much better than having to store global state somewhere else. Just make sure you measure the usage and chose a different design when you notice when this becomes a problem.
On the other hand, some of them do have performance problems and are a nightmare to migrate to a different solution, it's hard to reason about what parts of the application turn out to be stateful.
I guess it depends on what world you live in. For example, using ASPNET Core, I just drop in this https://learn.microsoft.com/en-us/aspnet/core/performance/ra... and boom I have rate limiting and I do not have to stress about threads or state or whatever.