Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am building a cloud

crawshaw.io

Post reply on HN
Print view View original

131–140 of 589 posts

Re: I am building a cloud

#131

Post by sahil-shubham »

The point about VMs being the wrong shape because they’re tied to CPU/memory resonates hard. The abstraction forces you to pay for time, not work.

I ended up buying a cheap auctioned Hetzner server and using my self-hostable Firecracker orchestrator on top of it (https://github.com/sahil-shubham/bhatti, https://bhatti.sh) specifically because I wanted the thing he’s describing — buy some hardware, carve it into as many VMs as I want, and not think about provisioning or their lifecycle. Idle VMs snapshot to disk and free all RAM automatically. The hardware is mine, the VMs are disposable, and idle costs nothing.

The thing that, although obvious, surprised me most is that once you have memory-state snapshots, everything becomes resumable. I make a browser sandbox, get Chromium to a logged-in state, snapshot it, and resume copies of that session on demand. My agents work inside sandboxes, I run docker compose in them for preview environments, and when nothing’s active the server is basically idle. One $100/month box does all of it.

Re: I am building a cloud

#132

Post by celrenheit »

chimpanzee2 wrote: post #125
celrenheit wrote: post #91

Shameless plug: https://clawk.work/ `ssh you/repo/branch@box.clawk.work` → jump directly into Claude Code (or Codex) with your repo cloned and credentials injected. Firecracker VMs, 19€/mo. POC, please be kind.

honestly sounds interesting at 19€/mo are you subsidizing it given the sharp rise of LLM costs lately? or are you heavily restricting model access. surely there is no Opus?

The 19€/mo is infra only. Claude Code inside the VM signs in via OAuth to the user's own Anthropic account. I'd love to explore bundling open models (Qwen, etc..) into the subscription down the line, but that needs product validation first, not going to ship something I'm not sure people actually want.

Re: I am building a cloud

#133

Post by goombaskoop »

dajonker wrote: post #93
adamtulinius wrote: post #89

Earlier quoted context omitted.

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

I totally agree, but that's not what happens in reality: the average devops knows k8s and will slap it onto anything they see (if only so they can put in on their resume). The average manager hears about k8s, gets convinced they need and hires beforementioned devops to build it.

> the average devops knows k8s and will slap it onto anything they see

This is certainly the case from all the third person accounts I hear. Online. I never actually met a single one that is like that, if anything, those same people are the ones that are first to tell me about their Hetzner setups.

Re: I am building a cloud

#134

Post by tee-es-gee »

I will follow this one for sure. There are a few more companies with the extremely ambitious goal of "a better AWS", and I am interested in the various strategies they take to approach that goal incrementally.

A service offering VMs for $20 is a long way from AWS, but I see how it makes sense as a first step. AWS also started with EC2, but in a completely different environment with no competition.

Re: I am building a cloud

#135

Post by zuhsetaqi »

faangguyindia wrote: post #68
stingraycharles wrote: post #2

Potentially useful context: OP is one of the cofounders of Tailscale. > Traditional Cloud 1.0 companies sell you a VM with a default of 3000 IOPS, while your laptop has 500k. Getting the defaults right (and the cost of those defaults right) requires careful thinking through the stack. I wish them a lot of luck! I admire the vision and am definitely a target customer, I'm just afraid this goes the way things always go…

i was just curious so i tested this actually. Using fio Hetzner (cx23, 2vCPU, 4 GB) ~3900 IOPS (read/write) ~15.3 MB/s avg latency ~2.1 ms 99.9th percentile ≈ ~5 ms max ≈ ~7 ms DigitalOcean (SFO1 / 2 GB RAM / 30 GB Disk) ~3900 IOPS (same!) ~15.7 MB/s (same!) avg latency ~2.1 ms (same!) 99.9th percentile ≈ ~18 ms max ≈ ~85 ms (!!) using sequential dd Hetzner: 1.9 GB/s DO: 850 MB/s Using low end plan on both but this H…

Just for comparison I use the cheapest netcup root server:

RS 1000 G12 AMD EPYC™ 9645 8 GB DDR5 RAM (ECC) 4 dedicated cores 256 GB NVMe

Costs 12,79 €

Results with the follwing command:

fio --name=randreadwrite \ --filename=testfile \ --size=5G \ --bs=4k \ --rw=randrw \ --rwmixread=70 \ --iodepth=32 \ --ioengine=libaio \ --direct=1 \ --numjobs=4 \ --runtime=60 \ --time_based \ --group_reporting

IOPS Read: 70.1k IOPS Write: 30.1k IOPS ~100k IOPS total

Throughput Read: 274 MiB/s Write: 117 MiB/s

Latency Read avg: 1.66 ms, P99.9: 2.61 ms, max 5.644 ms Write avg: 0.39 ms, P99.9: 2.97 ms, max 15.307 ms

Re: I am building a cloud

#136

Post by mystifyingpoi »

dgb23 wrote: post #119
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

> Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined networking layer on top of it. I'm not familiar with kubernetes, but doesn't it already do SDN out of the box?

> doesn't it already do SDN out of the box

Yes and no. Kubernetes defines specification about network behavior (in form of CNI), but it contains no actual implementation. You have to install the network plugin basically as the first setup step.

Re: I am building a cloud

#137

Post by Thanemate »

adamtulinius wrote: post #89
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

I know that "resume-driven development" exists, where the tradeoffs between approaches aren't about the technical fit of the solution but the career trajectory. I've seen people making plain workstation preparation scripts using Rust, only to have something to flex about in interviews.

I'm not surprised even in the slightest that DevOps workers will slap k8s on everything, to show "real industry experience" in a job market where the resume matches the tools.

Re: I am building a cloud

#138

Post by onlybosshaskeys »

eddythompson80 wrote: post #112
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

And those devops folks just let your single debian VM be? It sounds like you have, like many of us, an organizational/people problem, not a k8s problem. Maybe those devops folks only pay attention to k8s clusters and you're flying under their radar with your single debian VM + Kamal. But the same thinking that results in an overtly complex, impossible to debug, expensive to run k8s cluster can absolutely result in th…

I mean, it's pretty clear the only reason they even got to swap to a single VM and take the glory is because they fired the devops in question. As in, they're the actual boss of a small operation. That's what saying goodbye and nuking the cluster implies here.

Re: I am building a cloud

#139

Post by effisfor »

qxmat wrote: post #127

Europe is crying out for sovereign clouds. If this is to be a viable alt cloud, US jurisdiction is a no. Not sure we can move away from cpu/memory/io budgeting towards total metal saturation because code isn't what it used to be because no one handles malloc failure any more, we just crash OOM

For anybody interested, the meat of 'EU sovereign' means EU companies, not US or UK companies with EU servers. (because of CLOUD Act and the UK-US bilateral arrangement connected to it).

International visitors might tell us more about benefits of non EU, US or UK nexus companies/legal/rights.

Post reply on HN