I'm coming around to building everything for a VPS from the outset. There's a lot of upside to VPSes, such as:
1. Can be purchased as a fixed cost, usually at a rate that's much cheaper than on-demand pricing, and especially serverless--this tends to only get better with time as competition keeps prices low
2. It's "just" a Unix/Windows/Mac box, so the issues with runtime constraints you mention are bounded differently (and often more favorably); serverless is also just a box, but the constraints tend to be more onerous and limiting and it's not usually accessible in the same way
3. With containers, it's trivial to move between providers, so the hardware itself becomes fungible
4. On containers, I'm having a great time shipping Docker Compose configs--this works really well for the scale of application I'm targeting while avoiding the dreaded complexity of e.g. k8s
5. There's decades of high quality tooling already built and battle tested which makes operating VPSes much easier; the fact you can SSH into the machine, for instance, has huge leverage as an solo person working on independent products
Going forward, I'm planning to skip edge compute altogether unless there's a really compelling reason to want it. I should also mention that when a VPS is paired with a CDN, you can layer on bits of "edge compute" where it's warranted; or, you know, use it to cache static assets close to your users. :)
All-in-all it's kind of a funny return to where I started ~20 years or so ago with web development.