Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use One Big Server (2022)

specbranch.com

Post reply on HN
Print view View original

101–110 of 330 posts

Re: Use One Big Server (2022)

#101

Post by vevoe »

joshmn wrote: post #35
vevoe wrote: post #33

Earlier quoted context omitted.

Is there somewhere I can read more about your setup/experience with your streaming site? I currently run a (legal :) streaming site but have it hosted on AWS and have been exploring moving everything over to a big server. At this point it just seems like more work to move it than to just pay the cloud tax.

Do a search for HeheStreams on your favorite search engine. The technical bits aren’t all there, though, and there’s a plethora of noise and misinformation. Happy to talk via email though.

Will do, thank you!

Re: Use One Big Server (2022)

#102

Post by themafia »

matt-p wrote: post #90
themafia wrote: post #88

Earlier quoted context omitted.

On AWS if you want raw computational capacity you use Lambda and not EC2. EC2 is for legacy type workloads and doesn't have nearly the same scaling power and speed that Lambda does. I have several workloads that just invoke Lambda in parallel. Now I effectively have a 1000 core machine and can blast through large workloads without even thinking about it. I have no VM to maintain or OS image to consider or worry about…

Very few providers charge setup, some will provision a server within a 90s of an api call.

Hertzner does on the server the OP was referencing:

https://www.hetzner.com/dedicated-rootserver/ax162-s/

Re: Use One Big Server (2022)

#103

Post by themafia »

icedchai wrote: post #99
themafia wrote: post #88

Earlier quoted context omitted.

On AWS if you want raw computational capacity you use Lambda and not EC2. EC2 is for legacy type workloads and doesn't have nearly the same scaling power and speed that Lambda does. I have several workloads that just invoke Lambda in parallel. Now I effectively have a 1000 core machine and can blast through large workloads without even thinking about it. I have no VM to maintain or OS image to consider or worry about…

That's fine, except for all of Lambda's weird limitations: request and response sizes, deployment .zip sizes, max execution time, etc. For anything complicated you'll eventually you run into all this stuff. Plus you'll be locked into AWS.

> request and response sizes

If either of these exceed the limitations of the call, which is 6MB or 256kB depending on call type, then you can just use S3. For large distributed task coordination you're going to be doing this anyways.

> deployment .zip sizes

Overlays exist and are powerful.

> max execution time

If your workload depends on long uninterrupted runs of time on single CPUs then you have other problems.

> Plus you'll be locked into AWS.

In the world of serverless your interface to the endpoints and semantics of Lambda are minimal and easily changed.

Re: Use One Big Server (2022)

#104

Post by themafia »

solid_fuel wrote: post #92
themafia wrote: post #88

Earlier quoted context omitted.

On AWS if you want raw computational capacity you use Lambda and not EC2. EC2 is for legacy type workloads and doesn't have nearly the same scaling power and speed that Lambda does. I have several workloads that just invoke Lambda in parallel. Now I effectively have a 1000 core machine and can blast through large workloads without even thinking about it. I have no VM to maintain or OS image to consider or worry about…

Strong disagree here. Lambda is significantly more expensive per vCPU hour and introduces tight restrictions on your workflow and architecture, one of the most significant being maximum runtime duration. Lambda is a decent choice when you need fast, spiky scaling for a lot simple self-contained tasks. It is a bad choice for heavy tasks like transcoding long videos, training a model, data analysis, and other compute-h…

> significantly more expensive per vCPU hour

It's almost exactly the same price as EC2. What you don't get to control is the mix of vCPU and RAM. Lambda ties those two together. For equivalent EC2 instances the cost difference is astronomically small, on the order of pennies per month.

> like transcoding long videos, [...] data analysis, and other compute-heavy tasks

If you aren't breaking these up into multiple smaller independent segments then I would suggest that you're doing this wrong in the first place.

> training a model

You're going to want more than what a basic EC2 instance affords you in this case. The scaling factors and velocity are far less of a factor.

Re: Use One Big Server (2022)

#105

Post by vidarh »

bspammer wrote: post #84
matt-p wrote: post #74

Earlier quoted context omitted.

The difference between a fairly expensive ($300) RDS instance + EC2 in the same region vs a $90 dedicated server with a NVME drive and postgres in a container is absolutely insane.

A fair comparison would include the cost of the DBA who will be responsible for backups, updates, monitoring, security and access control. That’s what RDS is actually competing with.

I do consulting in this space, and we consistently make more money from people who insist on using cloud services, because their setups tend to need far more work.

Re: Use One Big Server (2022)

#106

Post by sgarland »

bspammer wrote: post #84
matt-p wrote: post #74

Earlier quoted context omitted.

The difference between a fairly expensive ($300) RDS instance + EC2 in the same region vs a $90 dedicated server with a NVME drive and postgres in a container is absolutely insane.

A fair comparison would include the cost of the DBA who will be responsible for backups, updates, monitoring, security and access control. That’s what RDS is actually competing with.

You don’t need a DBA for any of those, you need someone who can read some docs. It’s not witchcraft.

Re: Use One Big Server (2022)

#107

Post by sgarland »

dardeaup wrote: post #100
bob1029 wrote: post #10

This isn't even the end game for "one big server". AMD will give the most bang per rack, but there are other factors. An IBM z17 is effectively one big server too, but provides levels of reliability that are simply not available in most IT environments. It won't outperform the AMD rack, but it will definitely keep up for most practical workloads. If you sit down and really think honestly about the cost of engineering…

At what cost politically? I would expect political battles to be far more intense than any of the technical ones.

That’s because 75% (citation: wild-ass estimate) of tech workers are incapable of critical thinking, and blindly parrot whatever they’ve heard / read. The number of times I’ve seen something on HN, thought “that doesn’t sound right,” and then spent a day disproving it locally is too damn high. Of course, by then no one gives a shit, and they’ve all moved on patting each other on the back about how New Shiny is better.

Re: Use One Big Server (2022)

#108

Post by vidarh »

SatvikBeri wrote: post #96

A lot of these articles look at on-demand pricing for AWS. But you're rarely paying on-demand prices 24/7. If you have a stable workload, you probably buy reserved instances or a compute savings plan. At larger scales, you use third party services to get better deals with more flexibility. A while back I looked into renting hardware, and found that we would save about 20% compared to what we actually paid AWS – in pa…

This is usually only true of you lift and shift your AWS setup exactly as-is, instead of looking at what hardware will run your setup most efficiently.

The biggest cost with AWS also isn't compute, but egress - for bandwidth heavy setups you can sometimes finance the entirety of the servers from a fraction of the savings in egress.

I cost optimize setups with guaranteed caps at a proportion of savings a lot of the time, and I've yet to see a setup where we couldn't cut the cost far more than that.

Re: Use One Big Server (2022)

#109

Post by vidarh »

Havoc wrote: post #79

>Unfortunately, since all of your services run on servers (whether you like it or not), someone in that supply chain is charging you based on their peak load. This seems fundamentally incorrect to me? If I need 100 units of peak compute during 8 hours of work hours, I get that from Big Cloud, and they have two other clients needing same in offset timezones then in theory the aggregate cost of that is 1/3rd of everyon…

But they generally don't. Most people don't have large enough daily fluctuations for these demand curves to flatten out enough. And the providers also need enough capacity to handle unforeseen spikes. Which is also why none of them will let you scale however far you want - they still impose limits so they can plan the excess they need.

Re: Use One Big Server (2022)

#110

Post by sgarland »

Puts wrote: post #11
talles wrote: post #3

Don't forget the cost of managing your one big server and the risk of having such single point of failure.

My experience after 20 years in the hosting industry is that customers in general have more downtime due to self-inflicted over-engineered replication, or split brain errors than actual hardware failures. One server is the simplest and most reliable setup, and if you have backup and automated provisioning you can just re-deploy your entire environment in less than the time it takes to debug a complex multi-server set…

Yep. I know people will say, “it’s just a homelab,” but hear me out: I’ve ran positively ancient Dell R620s in a Proxmox cluster for years. At least five. Other than moving them from TX to NC, the cluster has had 100% uptime. When I’ve needed to do maintenance, I drop one at a time, and it maintains quorum, as expected. I’ll reiterate that this is on circa-2012 hardware.

In all those years, I’ve had precisely one actual hardware failure: a PSU went out. They’re redundant, so nothing happened, and I replaced it.

Servers are remarkably resilient.

EDIT: 100% uptime modulo power failure. I have a rack UPS, and a generator, but once I discovered the hard way that the UPS batteries couldn’t hold a charge long enough to keep the rack up while I brought the generator online.

Post reply on HN