Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use One Big Server (2022)

specbranch.com

Post reply on HN
Print view View original

61–70 of 330 posts

Re: Use One Big Server (2022)

#61

Post by KronisLV »

Just today I wasted some time due to an unexpected Tailscale key expiry and some other issues related to running a container cluster: https://blog.kronis.dev/blog/the-great-container-crashout

Right now, my plan is to move from a bunch of separate VPSes, to one dedicated server from Hetzner and run a few VMs inside of it with separate public IPs assigned to them alongside some resource limits. You can get them for pretty affordable prices, if you don't need the latest hardware: https://www.hetzner.com/sb/

That way I can limit the blast range if I mess things up inside of a VM, but at the same time benefit from an otherwise pretty simple setup for hosting personal stuff, a CPU with 8 threads and 64 GB of RAM ought to be enough for most stuff I might want to do.

Re: Use One Big Server (2022)

#62

Post by runako »

Demiurge wrote: post #53
runako wrote: post #26

One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider. Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago. At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstat…

I think it’s the other way around. I’m a huge fan of Hetzner for small sites with a few users. However, for bigger projects, the cloud seems to offer a complete lack of constraints. For projects that can pay for my time, $200/m or $2000/m in hosting costs is a negligible difference. What’s the development cost difference between AWS CDK / Terraform + GitHub Actions vs. Docker / K8s / Ansible + any CI pipeline? I don’…

> I really don't see how "bare metal" saves any engineering time

This is because you are looking only at provisioning/deployment. And you are right -- node size does not impact DevOps all that much.

I am looking at the solution space available to the engineers who write the software that ultimately gets deployed on the nodes. And that solution space is different when the nodes have 10x the capability. Yes, cloud providers have tons of aggregate capability. But designing software to run on a fleet of small machines is very different from accomplishing the same tasks on a single large machine.

It would not be controversial to suggest that targeting code at an Apple Watch or Raspberry Pi imposes constraints on developers that do not exist when targeting desktops. I am saying the same dynamic now applies to targeting cloud providers.

This isn't to say there's a single best solution for everything. But there are tradeoffs that are now always apparent. The art is knowing when it makes sense to pay the Cloud Tax, and whether to go 100% Cloud vs some proportion of dedicated.

Re: Use One Big Server (2022)

#63

Post by railorsi »

matt-p wrote: post #6

A thoroughly good article. It's probably worth also considering adding a CDN if you take this approach at scale. You get to use their WAF and DNS failover. A big pain point that I personally don't love is that this non-cloud approach normally means running my own database. It's worth considering a provider who also provides cloud databases. If you go for an 'active/passive' setup, consider saving even more money by u…

What’s the issue with running Postgres inside a docker container + regular backups? Never had problem and relatively easy to manage.

Re: Use One Big Server (2022)

#64

Post by justmarc »

talles wrote: post #3

Don't forget the cost of managing your one big server and the risk of having such single point of failure.

AWS has also been a single point of failure multiple times in history, and there's no reason to believe this will never happen again.

Re: Use One Big Server (2022)

#66

Post by johnklos »

These days we have more meta-software than software. Instead of Apache with virtualhosts, we have a VM running Docker instances, each with an nginx of its own, all connected by a separate Docker of nginx acting as a proxy.

How much waste is there from all this meta-software?

In reality, I host more on Raspberry Pis with USB SSDs than some people host on hundred-plus watt Dells.

At the same time, people constantly compare colo and hardware costs with the cost per month of cloud and say cloud is "cheaper". I don't even bother to point out the broken thinking that leads to that. In reality, we can ignore gatekeepers and run things out of our homes, using VPSes for public IPs when our home ISPs won't allow certain services, and we can still have excellent uptimes, often better than cloud uptimes.

Yes, we can consolidate many, many services in to one machine because most services aren't resource heavy constantly.

Two machines on two different home ISP networks backing each other up can offer greater aggregate uptime than a single "enterprise" (a misnomer, if you ask me, if you're talking about most x86 vendors) server in colo. A single five minute reboot of a Dell a year drops uptime from 100% to 99.999%.

Cloud is such bullshit that it's exhausting even just engaging with people who "but what if" everything, showing they've never even thought about it for more than a minute themselves.

Re: Use One Big Server (2022)

#67

Post by cpursley »

fragmede wrote: post #44
DaSHacka wrote: post #40

Earlier quoted context omitted.

> Or, play with servers and K8s, burning your investors money, then move on to the next gig and repeat... I mean, of the two, the PaaS route certainly burns more money, the exception being the rare shop that is so incompetent they can't even get their own infrastructure configured correctly, like in GP's situation. There are guaranteed more shops that would be better off self-hosting and saving on their current massi…

> the PaaS route certainly burns more money, Does it? Vercel is $20/month and Neon starts at $5/month. That obviously goes up as you scale up, but $25/month seems like a fairly cheap place to start to me. (I don't work for Vercel or Neon, just a happy customer)

Yeah, also a happy neon customer - but they can get pricy. Still prefer them over AWS. For compute, Fly is pretty competitive.

Re: Use One Big Server (2022)

#68

Post by marcosdumay »

alkonaut wrote: post #57

Microservices vs not is (almost) orthogonal to N servers vs one. You can make 10 microservices and rent a huge server and run all 10 services. It's more an organizational thing than a deployment thing. You can't do the opposite though, make a monolith and spread it out on 10 servers.

> You can't do the opposite though, make a monolith and spread it out on 10 servers.

You absolutely can, and it has been the most common practice for scaling them for decades.

Re: Use One Big Server (2022)

#69

Post by shrubble »

runako wrote: post #26

One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider. Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago. At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstat…

It's more than that - it's all the latency that you can remove from the equation with your bare-metal server.

No network latency between nodes, less memory bandwidth latency/contention as there is in VMs, no caching architecture latency needed when you can just tell e.g. Postgres to use gigs of RAM and then let Linux's disk caching take care of the rest (and not need a separate caching architecture).

Re: Use One Big Server (2022)

#70

Post by Demiurge »

runako wrote: post #62
Demiurge wrote: post #53

Earlier quoted context omitted.

I think it’s the other way around. I’m a huge fan of Hetzner for small sites with a few users. However, for bigger projects, the cloud seems to offer a complete lack of constraints. For projects that can pay for my time, $200/m or $2000/m in hosting costs is a negligible difference. What’s the development cost difference between AWS CDK / Terraform + GitHub Actions vs. Docker / K8s / Ansible + any CI pipeline? I don’…

> I really don't see how "bare metal" saves any engineering time This is because you are looking only at provisioning/deployment. And you are right -- node size does not impact DevOps all that much. I am looking at the solution space available to the engineers who write the software that ultimately gets deployed on the nodes. And that solution space is different when the nodes have 10x the capability. Yes, cloud prov…

Overall, I agree that most people underestimate the runway that the modern dedicated server can give you.
Post reply on HN