Live data from Hacker News

I am building a cloud

crawshaw.io

161–170 of 589 posts

Re: I am building a cloud

#161

Earlier quoted context omitted.

Just to spell this out more clearly for the back row.of the classroom: The price is what the customer will pay, regardless of your costs.

Economics teaches us that a big difference between cost and price attracts competition which should make the price trend towards the cost.

Only if the barrier of entry is low.

Which it won't be, if at every turn you choose the hyperscaler.

Re: I am building a cloud

#162

i just use Hetzner. Everything which cloud companies provide just cost so much, my own postgres running with HA setup and backup cost me 1/10th the price of RDS or CloudSQL service running in production over 10 years with no downtime. i directly autoscales instances off of the Metrics harvested from graphana it works fine for us, we've autoscaler configured via webhooks. Very simple and never failed us. i don't know…

I find it interesting that Hetzner was never a consideration, until... LLMs started recommending them.

Re: I am building a cloud

#163

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

I thought we collectively learned this with stack overflows engineering blog years ago. Scale vertically until you can't because you're unlikely to hit a limit and if you do you'll have enough money to pay someone else to solve it. Docker is amazing development tooling but it makes for horrible production infrastructure.

Docker is great development tooling (still some rough edges, of course).

Docker Compose is good for running things on a single server as well.

Docker Swarm and Hashicorp Nomad are good for multi-server setups.

Kubernetes is... enterprise and I guess there's a scale where it makes sense. K3s and similar sort of fill the gap, but I guess it's a matter of what you know and prefer at that point.

Throw on Portainer on a server and the DX is pretty casual (when it works and doesn't have weird networking issues).

Of course, there's also other options for OCI containers, like Podman.

Re: I am building a cloud

#164
post #49

> Agents, by making it easiest to write code, means there will be a lot more software. Economists would call this an instance of Jevons paradox. Each of us will write more programs, for fun and for work. There is already so much software out there, which isn't used by anyone. Just take a look at any appstore. I don't understand why we are so obsessed with cranking out even more, whereas the obvious usecase for LLMs s…

That's not what Jevons paradox means though. He's just name dropping some concept.

Jevons paradox would be if despite software becoming cheaper to produce the total spend on producing software would increase because the increase in production outruns the savings

Jevons paradox applies when demand is very elastic, i.e. small changes in price cause large changes in quantity demanded. It's a property of the market.

Re: I am building a cloud

#165
I think clouds pay a huge abstraction penalty to allow tiny VMs. I guess it helps with onboarding and $10 personal VPNs. But I have never needed a fraction of a computer. I want to rent some number of full computers of various sizes, consisting of CPU, memory, and flash disk. Hetzner is closer than AWS, and I think/hope that’s what Crawshaw is aiming for.

Re: I am building a cloud

#166
post #24

Earlier quoted context omitted.

Agree, I used to always use Heroku or Render style platforms for my own software, but nowadays I just have a Linux server with Docker Compose and a Cron job. The cron job every minute runs docker pull (downloads latest image) and docker up -d (switches to new version only if there is a new version). And put caddy in front for the HTTPS. This has been very cheap and reliable for years now.

One annoyance (I don't know if they've since fixed it) was that Docker Hub would count pulls that don't contain an update towards the rate limit. That ultimately prompted me to switch to alternate repositories.

one way is to host a manifest file (can host one on r2) and update it on each deploy and when manifest changes, new container image is pulled.

Re: I am building a cloud

#167

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

A single VM is indeed the most pragmatic setup that most apps really need. However I still prefer to have at least two for little redundancy and peace of mind. It’s just less stressful to do any upgrades or changes knowing there is another replica in case of a failure. And I’m building and happily using Uncloud ( https://github.com/psviderski/uncloud ) for this (inspired by Kamal). It makes multi-machine setups as si…

That looks pretty interesting. Is it being used in production yet (I mean serious installs) ?

Re: I am building a cloud

#168

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

> It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined networking layer on top of it.

As a devops/cloud engineer coming from a pure sysadmin background (you've got a cluster of n machines running RHEL and that's it) i feel this.

The issues i see however are of different nature:

1. resumeè-driven development (people get higher-paying job if you have the buzzwords in your cv)

2. a general lack of core-linux skills. people don't actually understand how linux and kubernetes work, so they can't build the things they need, so they install off-the-shelf products that do 1000 things including the single one they need.

3. marketing, trendy stuff and FOMO... that tell you that you absolutely can't live without product X or that you must absolutely be doing Y

to give you an example of 3: fluxcd/argocd. they're large and clunky, and we're getting pushed to adopt that for managing the services that we run inside the cluster (not developer workloads, but mostly-static stuff like the LGTM stack and a few more things - core services, basically). they're messy, they add another layer of complexity, other software to run and troubleshoot, more cognitive load.

i'm pushing back on that, and frankly for our needs i'm fairly sure we're better off using terraform to manage kubernetes stuff via the kubernetes and helm provider. i've done some tests and frankly it works beautifully.

it's also the same tool we use to manage infrastructure, so we get to reuse a lot of skills we already have.

also it's fairly easy to inspect... I'm doing some tests using https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclparse and i'm building some internal tooling to do static analysis of our terraform code and automated refactoring.

i still think kubernetes is worth the hassle, though (i mostly run EKS, which by the way has been working very good for me)

Re: I am building a cloud

#169

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

yeah it's like wanting to drive to the mall in the Space Shuttle and then complaining how its too complicated

Re: I am building a cloud

#170

Potentially useful context: OP is one of the cofounders of Tailscale. > Traditional Cloud 1.0 companies sell you a VM with a default of 3000 IOPS, while your laptop has 500k. Getting the defaults right (and the cost of those defaults right) requires careful thinking through the stack. I wish them a lot of luck! I admire the vision and am definitely a target customer, I'm just afraid this goes the way things always go…

>3000 IOPS If that's true, I wonder if this is a deliberate decision by cloud providers to push users towards microservice architectures with proprietary cloud storage like S3, so you can't do on-machine dbs even for simple servers.

It's probably a combination of high density storage nodes getting I/O bound and SSDs having finite write endurance. Anything that improves the first problem costs them money to improve it and then makes the second problem worse, and the second one costs them money again, so why would they want to make the default something that costs then more twice if most people don't need it?

Instead they make the default "meager IOPS" and then charge more to the people who need more.

Post reply on HN