For hobby projects, I own a moderately outdated 1U "pizzabox" installed in a downmarket colocation facility in a nearby major city. Considering the monthly colocation rate and the hardware cost amortized over two years (I will probably not replace it that soon but it's what I've used for planning), this works out to appreciably less than it would cost to run a similar workload on a cloud provider. It costs about the same or possibly a bit less than running the same workload on a downmarket dedi or VPS provider, but it feels more reliable (at least downtime is usually my fault and so under my control) and the specs on the machine are higher than it's easy to get from downmarket VPS operations.
Because my application involves live video transcoding I'm fairly demanding on CPU time, which is something that's hard to get (reliably) from a downmarket VPS operation (even DO or what have you) and costly from a cloud provider. On the other hand, dual 8 core Xeons don't cost very much when they're almost a decade old and they more than handle the job.
There are a few fairly reputable vendors for used servers out there, e.g. Unix Surplus, and they're probably cheaper than you think. I wouldn't trust used equipment with a business-critical workload but honestly it's more reliable than an EC2 instance in terms of lifetime-before-unscheduled-termination, and since I spend my workday doing "cloud-scale" or whatever I have minimal interest in doing it in my off-time, where I prefer to stick to an "old fashioned" approach of keeping my pets fed and groomed.
And, honestly, new equipment is probably cheaper than you think. Dealing with a Dell account rep is a monumental pain but the prices actually aren't that crazy. Last time I purchased over $100k in equipment (in a professional context, my hobbies haven't gotten that far yet) I was able to get a lot for it - and that's well less than burdened cost for one engineer.