Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use One Big Server (2022)

specbranch.com

Post reply on HN
Print view View original

51–60 of 330 posts

Re: Use One Big Server (2022)

#51

Post by andersmurphy »

runako wrote: post #26

One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider. Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago. At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstat…

100% this add an embedded database like sqlite and optimise writes to batch and you can go really really far with hetzner. It's also why I find the "what about overprovisioning" argument silly (once you look outside of AWS you can get insane cost/perf ratio).

Also in my experience more complex systems tend to have much less reliability/resilience than simple single node systems. Things rarely fail in isolation.

Re: Use One Big Server (2022)

#52

Post by jeffrallen »

I work for a cloud provider and I'll tell you, one of the reasons for the cloud premium is that it is a total pain in the ass to run hardware. Last week I installed two servers and between them had four mysterious problems that had to be solved by reseating cards, messing with BIOS settings, etc. Last year we had to deal with a 7 site, 5 country RMA for 150 100gb copper cables with incorrect coding in their EEPROMs.

I tell my colleagues: it's a good thing that hardware sucks: the harder it is to run bare metal, the happier our customers are that they choose the cloud. :)

(But also: this is an excellent article, full of excellent facts. Luckily, my customers choose differently.)

Re: Use One Big Server (2022)

#53

Post by Demiurge »

runako wrote: post #26

One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider. Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago. At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstat…

I think it’s the other way around. I’m a huge fan of Hetzner for small sites with a few users. However, for bigger projects, the cloud seems to offer a complete lack of constraints. For projects that can pay for my time, $200/m or $2000/m in hosting costs is a negligible difference. What’s the development cost difference between AWS CDK / Terraform + GitHub Actions vs. Docker / K8s / Ansible + any CI pipeline? I don’t know; in my experience, I don’t see how “bare metal” saves much engineering time. I also don’t see anything complicated about using an IaC Fargate + RDS template.

Now, if you actually need to decouple your file storage and make it durable and scalable, or need to dynamically create subdomains, or any number of other things… The effort of learning and integrating different dedicated services at the infrastructure level to run all this seems much more constraining.

I’ve been doing this since before the “Cloud,” and in my view, if you have a project that makes money, cloud costs are a worthwhile investment that will be the last thing that constrains your project. If cloud costs feel too constraining for your project, then perhaps it’s more of a hobby than a business—at least in my experience.

Just thinking about maintaining multiple cluster filesystems and disk arrays—it’s just not what I would want to be doing with most companies’ resources or my time. Maybe it’s like the difference between folks who prefer Arch and setting up Emacs just right, versus those happy with a MacBook. If I felt like changing my kernel scheduler was a constraint, I might recommend Arch; but otherwise, I recommend a MacBook. :)

On the flip side, I’ve also tried to turn a startup idea into a profitable project with no budget, where raw throughput was integral to the idea. In that situation, a dedicated server was absolutely the right choice, saving us thousands of dollars. But the idea did not pan out. If we had gotten more traction, I suspect we would have just vertically scaled for a while. But it’s unusual.

Re: Use One Big Server (2022)

#54

Post by icedchai »

fogx wrote: post #34
motorest wrote: post #15

Earlier quoted context omitted.

> My experience after 20 years in the hosting industry is that customers in general have more downtime due to self-inflicted over-engineered replication, or split brain errors than actual hardware failures. I think you misread OP. "Single point of failure" doesn't mean the only failure modes are hardware failures. It means that if something happens to your nodes whether it's hardware failure or power outage or someon…

don't you think it's highly unlikely that someone will stumble over the power cable in a hosted datacenter like hetzner? and even if, you could just run a provisioned secondary server that jumps in if the first becomes unavailable and still be much cheaper.

It's unlikely, but it happens. In the mid 2000's I had some servers at a colo. They were doing electrical work and took out power to a bunch of racks, including ours. Those environments are not static.

Re: Use One Big Server (2022)

#55

Post by andersmurphy »

randomtoast wrote: post #50

Those servers are mainly designed for enterprise use cases. For hobby projects, I can understand why someone would choose Hetzner over AWS. For enterprise environments, however, there is much more to consider. One of the biggest costs you face is your operations team. If you go with Hetzner, you essentially have to rebuild a wide range of infrastructure components yourself (WAF, globally distributed CDN, EFS, RDS, EK…

Most of those things you listed are work arounds for having a slow server/system.

For example, if you serve your assets from the server you can skip a cors round trip. If you use an embedded database like sqlite you can shave off 50ms, use dedicated CPU (another 50ms), now you don't need to sever anything from the edge. Because your global latency is much better.

Managing a single VPS is trivial compared to AWS.

Re: Use One Big Server (2022)

#56

Post by icedchai »

fragmede wrote: post #47
garganzol wrote: post #43

And then boom, all your services are gone due to a pesky capacitor on the motherboard. Also good luck trying to change even one software component of that monolith without disrupting and jeopardizing the whole operation. While it is a useful advice to some people in certain conditions, it should be taken with a grain of salt.

That capacitor thing hasn't been true since the 90's.

Capacitor problem or not, hardware does fail. Power supplies crap out. SSDs die in strange ways. A failure of a supposedly "redundant" SSD might cause your system to freeze up.

Re: Use One Big Server (2022)

#57

Post by alkonaut »

Microservices vs not is (almost) orthogonal to N servers vs one. You can make 10 microservices and rent a huge server and run all 10 services. It's more an organizational thing than a deployment thing. You can't do the opposite though, make a monolith and spread it out on 10 servers.

Re: Use One Big Server (2022)

#58

Post by garganzol »

fragmede wrote: post #47
garganzol wrote: post #43

And then boom, all your services are gone due to a pesky capacitor on the motherboard. Also good luck trying to change even one software component of that monolith without disrupting and jeopardizing the whole operation. While it is a useful advice to some people in certain conditions, it should be taken with a grain of salt.

That capacitor thing hasn't been true since the 90's.

Hardware still fails. It isn't a question of "if", it's a question of "when". Nothing lasts forever, the naivety lasts only so long too.

Re: Use One Big Server (2022)

#59

Post by jeffrallen »

Puts wrote: post #11
talles wrote: post #3

Don't forget the cost of managing your one big server and the risk of having such single point of failure.

My experience after 20 years in the hosting industry is that customers in general have more downtime due to self-inflicted over-engineered replication, or split brain errors than actual hardware failures. One server is the simplest and most reliable setup, and if you have backup and automated provisioning you can just re-deploy your entire environment in less than the time it takes to debug a complex multi-server set…

Not to mention the other leading cause of outages: UPS's.

Sigh.

Re: Use One Big Server (2022)

#60

Post by toast0 »

fogx wrote: post #34
motorest wrote: post #15

Earlier quoted context omitted.

> My experience after 20 years in the hosting industry is that customers in general have more downtime due to self-inflicted over-engineered replication, or split brain errors than actual hardware failures. I think you misread OP. "Single point of failure" doesn't mean the only failure modes are hardware failures. It means that if something happens to your nodes whether it's hardware failure or power outage or someon…

don't you think it's highly unlikely that someone will stumble over the power cable in a hosted datacenter like hetzner? and even if, you could just run a provisioned secondary server that jumps in if the first becomes unavailable and still be much cheaper.

I don't know about Hetzner, but the failure case isn't usually tripping over power plugs. It's putting a longer server in the rack above/below yours and pushing the power plug out of the back of your server.

Either way, stuff happens, figuring out what your actual requirements around uptime, time to response, and time to resolution is important before you build a nine nines solution when eight eights is sufficient. :p

Post reply on HN