I am building a cloud • Hacker News Board

Re: I am building a cloud

#151

Post by Medowar » Thu, Apr 23, 2026, 9:00 AM UTC

zuhsetaqi wrote: post #135

faangguyindia wrote: post #68

Earlier quoted context omitted.

i was just curious so i tested this actually. Using fio Hetzner (cx23, 2vCPU, 4 GB) ~3900 IOPS (read/write) ~15.3 MB/s avg latency ~2.1 ms 99.9th percentile ≈ ~5 ms max ≈ ~7 ms DigitalOcean (SFO1 / 2 GB RAM / 30 GB Disk) ~3900 IOPS (same!) ~15.7 MB/s (same!) avg latency ~2.1 ms (same!) 99.9th percentile ≈ ~18 ms max ≈ ~85 ms (!!) using sequential dd Hetzner: 1.9 GB/s DO: 850 MB/s Using low end plan on both but this H…

Just for comparison I use the cheapest netcup root server: RS 1000 G12 AMD EPYC™ 9645 8 GB DDR5 RAM (ECC) 4 dedicated cores 256 GB NVMe Costs 12,79 € Results with the follwing command: fio --name=randreadwrite \ --filename=testfile \ --size=5G \ --bs=4k \ --rw=randrw \ --rwmixread=70 \ --iodepth=32 \ --ioengine=libaio \ --direct=1 \ --numjobs=4 \ --runtime=60 \ --time_based \ --group_reporting IOPS Read: 70.1k IOPS W…

That is a bit of a unfair comparison. The Hetzner and DO instances are shared hosting, you are using dedicated ressources.

Using a Netcup VPS 1000 G12 is more comparable.

read: IOPS=18.7k, BW=73.1MiB/s

write: IOPS=8053, BW=31.5MiB/s

Latency Read avg: 5.39 ms, P99.9: 85.4 ms, max 482.6 ms

Write avg: 3.36 ms, P99.9: 86.5 ms, max 488.7 ms

Re: I am building a cloud

#152

Post by RandomBK » Thu, Apr 23, 2026, 9:01 AM UTC

huijzer wrote: post #24

faangguyindia wrote: post #9

i just use Hetzner. Everything which cloud companies provide just cost so much, my own postgres running with HA setup and backup cost me 1/10th the price of RDS or CloudSQL service running in production over 10 years with no downtime. i directly autoscales instances off of the Metrics harvested from graphana it works fine for us, we've autoscaler configured via webhooks. Very simple and never failed us. i don't know…

Agree, I used to always use Heroku or Render style platforms for my own software, but nowadays I just have a Linux server with Docker Compose and a Cron job. The cron job every minute runs docker pull (downloads latest image) and docker up -d (switches to new version only if there is a new version). And put caddy in front for the HTTPS. This has been very cheap and reliable for years now.

One annoyance (I don't know if they've since fixed it) was that Docker Hub would count pulls that don't contain an update towards the rate limit. That ultimately prompted me to switch to alternate repositories.

Re: I am building a cloud

#153

Post by PunchyHamster » Thu, Apr 23, 2026, 9:01 AM UTC

dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

Well, you used a tank to plow a field then complained about maintenance and fuel usage.

If you have actual need to deploy few dozen services all talking with eachother k8s isn't bad way to do it, it has its problems but it allows your devs to mostly self-service their infrastructure needs vs having to process ticket for each vm and firewall rules they need. That is saying from perspective of migrating from "old way" to 14 node actual hardware k8s cluster.

It does make debugging harder as you pretty much need central logging solution, but at that scale you want central logging solution anyway so it isn't big jump, and developers like it.

Main problem with k8s is frankly nothing technical, just the "ooh shiny" problem developers have where they see tech and want to use tech regardless of anything

Re: I am building a cloud

#154

Post by piokoch » Thu, Apr 23, 2026, 9:04 AM UTC

How this is different from getting dedicated server from any other provider? Typically you need to pay a bit more - $40-$50 but you get more RAM and cores.

And what it has to do with the "cloud"? Cloud means one use cloud-provided services - security, queue, managed database, etc. and that's their selling point. This exe.dev is a bare server where I can install what I want, this is fine, but this is not a cloud and, frankly speaking, nothing new.

Re: I am building a cloud

#155

Post by RandomBK » Thu, Apr 23, 2026, 9:04 AM UTC

omnimus wrote: post #150

kippinsula wrote: post #108

Earlier quoted context omitted.

we've done both. Hetzner dedicated was genuinely fine, until a disk started throwing SMART warnings on a Sunday morning and we remembered why we pay 10x elsewhere for some things. probably less about the raw cost and more about which weekends you want back.

Isn't this nature of every dedicated server? You also take on the hardware management burden - that's why they can be insanely cheap. But there is middleground in form of VPS, where hardware is managed by the provider. It's still way way cheaper than some cloud magic service.

VPS comes at the cost of potential for oversubscription - even from more reputable vendors. You never really know if you're actually getting what you're paying for.

Re: I am building a cloud

#156

Post by omnimus » Thu, Apr 23, 2026, 9:07 AM UTC

bedstefar wrote: post #100

This looks like an excellent platform for running a "homelab" in the cloud (no, the irony is not lost on me) for lighter stuff like Readeck, Calibre-web, Immich. Maybe even Home Assistant too if we can find a way (Tailscale?) to get the mDNS/multicast traffic tunnelled.

With pricing 100gb/8usd Immich would be wildly uneconomical. Better to wait for upcoming immich hosting to support the project or use ente.io - those are 1tb/10usd.

Re: I am building a cloud

#157

Post by hkt » Thu, Apr 23, 2026, 9:08 AM UTC

goombaskoop wrote: post #133

dajonker wrote: post #93

Earlier quoted context omitted.

I totally agree, but that's not what happens in reality: the average devops knows k8s and will slap it onto anything they see (if only so they can put in on their resume). The average manager hears about k8s, gets convinced they need and hires beforementioned devops to build it.

> the average devops knows k8s and will slap it onto anything they see This is certainly the case from all the third person accounts I hear. Online. I never actually met a single one that is like that, if anything, those same people are the ones that are first to tell me about their Hetzner setups.

DevOps here.

The trouble is that we are literally expected to do this everywhere we go. I've personally advocated for approaches which use say, a pair of dedicated servers, or VMs as in GPs example. If you want it outside of AWS/GCP/Azure, you're regarded as a crazy person. If you don't adopt "best practices" (as defined by vendors) then management are scared. Management very often trust the sales and marketing departments of big vendors more than their own staff. Many of us have given up fighting this, because what it comes down to is a massive asymmetry of information and trust.

Re: I am building a cloud

#158

Post by Quothling » Thu, Apr 23, 2026, 9:08 AM UTC

qxmat wrote: post #127

Europe is crying out for sovereign clouds. If this is to be a viable alt cloud, US jurisdiction is a no. Not sure we can move away from cpu/memory/io budgeting towards total metal saturation because code isn't what it used to be because no one handles malloc failure any more, we just crash OOM

Europe is already moving into the EU cloud. Hetzner, OGH Cloud and so on as well as local data centers where partner companies set up own cloud with various things to rival office 365. So far it's mainly the public sector. My own city cut their IT budget by 70% by switching from Microsoft.

The key point is the partner companies. Almost nobody is actually running their own clouds the way they would with various 365 products, AWS or Azure. They buy the cloud from partners, similar to how they used to (and still do) buy solutions from Microsoft partners. So if you want to "sell cloud" you're probably going to struggle unless you get some of these onboard. Which again would probably be hard because I imagine a lot of what they sell is sort of a package which basically runs on VM's setup as part of the package that they already have.

Re: I am building a cloud

#159

Post by psviderski » Thu, Apr 23, 2026, 9:08 AM UTC

dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

A single VM is indeed the most pragmatic setup that most apps really need. However I still prefer to have at least two for little redundancy and peace of mind. It’s just less stressful to do any upgrades or changes knowing there is another replica in case of a failure.

And I’m building and happily using Uncloud (https://github.com/psviderski/uncloud) for this (inspired by Kamal). It makes multi-machine setups as simple as a single VM. Creates a zero-config WireGuard overlay network and uses the standard Docker Compose spec to deploy to multiple VMs. There is no orchestrator or control plane complexity. Start with one VM, then add another when needed, can even mix cloud VMs and on-prem.

Re: I am building a cloud

#160

Post by PunchyHamster » Thu, Apr 23, 2026, 9:13 AM UTC

The author seems to have no clue what is cloud problem, and what is k8s problem, and is blaming everything on k8s. The whole post reeks of ignorance. I have no love to k8s but he is just flat out putting out false information.

> Finally, clouds have painful APIs. This is where projects like K8S come in, papering over the pain so engineers suffer a bit less from using the cloud.

K8s's main function isn't to paint over existing cloud APIs, that is just necessity when you deploy it in cloud. On normal hardware it's just an orchestration layer, and often just a way to pass config from one app to another in structured format.

> But VMs are hard with Kubernetes because the cloud makes you do it all yourself with lumpy nested virtualization.

Man discovered system designed for containers is good with containers, not VMs. More news at 10

> Disk is hard because back when they were designing K8S Google didn’t really even do usable remote block devices, and even if you can find a common pattern among clouds today to paper over, it will be slow.

Ignorance. k8s have abstractions over a bunch of types of storage, for example using Ceph as backend will just use KVM's Ceph backend, no extra overhead. It also supports "oldschool" protocols used for VM storage like NFS or iSCSI. It might be slow in some cases for cloud if cloud doesn't provide enough control, but that's not k8s fault.

> Networking is hard because if it were easy you would private link in a few systems from a neighboring open DC and drop a zero from your cloud spend.

He mistakes cloud problems with k8s problems(again). All k8s needs is visibility between nodes. There are multiple providers to achieve that, some with zero tunelling, just routing. It's still complex, but no more than "run a routing daemon".

I expect his project to slowly reinvent cloud APIs and copying what k8s and other projects did once he starts hitting problems those solutions solved. And do it worse, because instead of researching of why and why not that person seems to want to throw everything out with learning no lessons.

Do not give him money