Live data from Hacker News

Use One Big Server (2022)

specbranch.com

261–270 of 330 posts

Re: Use One Big Server (2022)

#261

Earlier quoted context omitted.

I don't get why people are so hell-bent on going to AWS, for the most minor applications, without looking at simpler options! I am not even thousands km near the level of what you are doing, but my client was paying $100/m for an AWS server, SQS and S3 bucket, for a small PHP based web application that uses Amazon Seller API, Keepa API for the products he ships. Used MySQL for data storage. I implemented the whole th…

Without understanding the architecture and use case better, at first read, my gut says that isn’t an AWS problem - it sounds like a solutions architecture problem. There are cheaper ways of building that use case on AWS. Most AWS sticker shock I’ve seen results from someone who doesn’t really understand cloud trying to build on the cloud. Cost has to be designed in from the start (in addition to security, operational…

... you failed at reading comprehension?

My comment was not about using AWS is bad, it has its uses. My comment was about how in this instance it was simply not needed. And I even speculated when it might be needed.

To pick the correct tool for the job, is what, it means to be an Engineer, or a person with common sense. With experience, we can get over childish absolutions of a tool or service, and look at the broader aspects, unless, of course, we are expecting some kind of monetary gains.

Re: Use One Big Server (2022)

#262

Earlier quoted context omitted.

I have used Knownhost previously, it served me really well. Before that, I used to go for Linode, but I think they've become more pricey?

Linode was bought by Akamai. They immediately raised prices, and they have been, if anything, less reliable.

Ahh, yes, I remember now! I think it's almost 8 years now? Stopped using them after the buy out.

Too bad, actually, their service was pretty good.

Re: Use One Big Server (2022)

#263
Ah, the folksy wisdom of the armchair. Sounds convincing, doesn't it? I mean, it includes math! And prices! The quoted prices are more expensive for the cloud. And he makes folksy claims that make sense, like the "fragile complexity" of having "more than one computer". It makes sense! Right??

But is he right? How do we know? Well for starters, look at his CV. He has never managed servers for a living. The closest he's come is working on FPGAs. So what's he basing all these opinions on? Musings? Thoughts? Feelings? Hope?

He makes a couple claims which it isn't obvious are bunk, so I'll address them here, in reverse order.

"microservice architectures in general add a lot of overhead to a system for dubious gain when you are running on one big server" - Microservices architectures are not about overhead or efficiency. They are an attempt to use good software design principles to address Conway's Law. If you design the microservice correctly, you can enable many different groups in an organization to develop software independently, and come up with a highly effective and flexible organization and stable products. Proof? Amazon. But the caveat is, you have to design them correctly. Almost everyone fails at this.

"It's impossible to get the benefits of a CDN, both in latency improvements and bandwidth savings, with one big server" - This is so dumb I'm not sure I have to refute it? But, uh, no, CDNs absolutely give a heap of benefits whether you have 1 server or 1,000. And CloudFlare Free Plan is Free.

"My Workload is Really Bursty - Cloud away." - Unless your workload involves massive amounts of storage or ingress/egress and your profit margin tiny, in which case you may save more by building out a small fleet of unreliable poorly-maintained colocated servers (emphasis on may).

"The "high availability" architectures you get from using cloudy constructs and microservices just about make up for the fragility they add due to complexity. ... Remember that we are trying to prevent correlated failures. Cloud datacenters have a lot of parts that can fail in correlated ways. Hosting providers have many fewer of these parts. Similarly, complex cloud services, like managed databases, have more failure modes than simple ones (VMs)." - Argument from laziness, or ignorance? He's trying to say that because something is complex it's also less reliable. Which completely ignores the reliability engineering aspect of that complexity. You mitigate higher numbers of failure modes by designing the system to fail over reliably. And you also have warm bodies running around replacing the failing parts, which fights entropy. You don't get that in a single server; once your power supply, disk, motherboard, network interface, RAM, etc fails, and assuming your server has a redundant pair, you have a ticking clock to repair it until the redundant pair fails. How lucky do you feel? (oh, and you'll need downtime to repair it.)

As usual, the cloud costs quoted is MSRP, and if you're paying retail, you're a fool. Almost all cloud costs can be brought down from 25%-75%, spot instances are a fraction of the on-demand server cost, and efficient use of cheaper cloud services reduces your need to buy compute at all.

"The big drawback of using a single big server is availability. Your server is going to need downtime, and it is going to break. Running a primary and a backup server is usually enough, keeping them in different datacenters. A 2x2 configuration should appease the truly paranoid: two servers in a primary datacenter (or cloud provider) and two servers in a backup datacenter will give you a lot of redundancy. If you want a third backup deployment, you can often make that smaller than your primary and secondary." - Wait... so One Big Server isn't enough? Huh. So this was a clickbait article? I'm shocked!

"One Server (Plus a Backup) is Usually Plenty" - Plenty for what? I mean we haven't even talked system architecture or application design. But let's assume it's a single microservice that gets 1RPS. Is your backup server a hot spare, cold spare, or live mirror? If it's live, it's experiencing the same wear, meaning it will fail at about the same time. If it's hot, there's less wear, but it's still experiencing some. If it's cold, you get less wear, but you're less sure it'll boot up again. And then there's system configuration. The author mentions the "complexity" of managing a cluster, but actually it's less complex than managing just two servers. With a fleet of servers, you know you have to use automation, so you spend the time to automate their setup and run updates frequently. With a backup, you probably won't do any maintenance on the backup, and you definitely won't perform the same operations on the backup as the server. So the system state will drift wildly, and the backup's software will be useless. It would be better to just have it as spare part.

The author never talks about the true failure modes of "one big server". When parts start to need replacing, it's never cheap. Smart hands cost, cost of the parts+shipping, cost of the downtime. And often you'll find there are delays - delays in getting smart hands to actually repair it correctly, delays in shipping, delays in part ordering/availability. Running out of power, running out of space, temperatures too high, "flaky" parts you can't diagnose, backups and restores, datacenter issues, routing issues, backbone issues. You'll tell yourself these are "probably rare" - but these are all failure modes, and as the author tells us, you should be wary of lots of failure modes. And anecdotes will tell you somebody has run a server for 10 years with no issue, while another person had a server with 3 faults in a month. To say nothing of the need to run "burn-in" on a new server to discover faults once it's racked.

Go ahead and do whatever you want. Cloud, colo, one server, multiple. There will be failures and complexity no matter what. You want to tell yourself a comforting story that there is "one piece of advice" to follow, some black and white world where only one piece of folksy wisdom applies. But here's my folksy wisdom: design your application, design your system to fit it, try not to pinch every penny, build something, and become educated enough to know what problems to expect and how to deal with them. Or if not, pay someone who can, and listen to them.

Re: Use One Big Server (2022)

#264
Bare-metal servers sound super cheap when you look at the price tag, and yeah, you get a lot of raw power for the money. But once you’re in an enterprise setup, the real cost isn’t the hardware at all, it’s the people needed to keep everything running.

If you go this route, you’ve got to build out your own stack for security, global delivery, databases, storage, orchestration, networking ... the whole deal. That means juggling a bunch of different tools, patching stuff, fixing breakage at 3 a.m., and scaling it all when things grow. Pretty soon you need way more engineers, and the “cheap” servers don’t feel so cheap anymore.

Re: Use One Big Server (2022)

#265

Earlier quoted context omitted.

I don't get why people are so hell-bent on going to AWS, for the most minor applications, without looking at simpler options! I am not even thousands km near the level of what you are doing, but my client was paying $100/m for an AWS server, SQS and S3 bucket, for a small PHP based web application that uses Amazon Seller API, Keepa API for the products he ships. Used MySQL for data storage. I implemented the whole th…

Saving $75 a month at what cost in labour?

You actually save on labour. A VPS is a lot less work than anything involving AWS console.

Re: Use One Big Server (2022)

#266

Earlier quoted context omitted.

Strong disagree here. Lambda is significantly more expensive per vCPU hour and introduces tight restrictions on your workflow and architecture, one of the most significant being maximum runtime duration. Lambda is a decent choice when you need fast, spiky scaling for a lot simple self-contained tasks. It is a bad choice for heavy tasks like transcoding long videos, training a model, data analysis, and other compute-h…

> significantly more expensive per vCPU hour It's almost exactly the same price as EC2. What you don't get to control is the mix of vCPU and RAM. Lambda ties those two together. For equivalent EC2 instances the cost difference is astronomically small, on the order of pennies per month. > like transcoding long videos, [...] data analysis, and other compute-heavy tasks If you aren't breaking these up into multiple smal…

This is a great example of what I meant when I said that a part of the Cloud Tax is it constrains the solution space available to developers. In an era where one can purchase, off-the-shelf, a 256-core machine with terabytes of RAM, developers are still counting megabytes(!) of file sizes due to the constraints of AWS.

It should be obvious that this is not the best answer for all projects.

Re: Use One Big Server (2022)

#267
post #189

Earlier quoted context omitted.

I don't get why people are so hell-bent on going to AWS, for the most minor applications, without looking at simpler options! I am not even thousands km near the level of what you are doing, but my client was paying $100/m for an AWS server, SQS and S3 bucket, for a small PHP based web application that uses Amazon Seller API, Keepa API for the products he ships. Used MySQL for data storage. I implemented the whole th…

What are some cheaper and better hosting providers that you can recommend?

For bare metal I’ve been using tier.net to get 192 GB RAM, 4TB NVME and 32 cores for $219/mo.

Data centers all over the country and I get to locate under 10ms from my regional audience.

Just a data point if you want some bigger iron than a VM.

Re: Use One Big Server (2022)

#268
post #231

I've been doing hybrid colo+public cloud for over a decade and it's always been the most cost effective route at a certain scale. That specific break even point is lowering over time with the density and cost effectiveness of hardware. Sure you need net/infra admins but the software and hardware these days are pretty management friendly and you'll find you still need (often more expensive "cloud") admins so you're no…

Colocation to me means you buy your own hardware and rent only the rack space (and power and connectivity) from the datacenter. Is that really what you're talking about? If so, why do you choose this over renting bare metal servers?

Not always - you can lease your servers from the vendor as well, in which case you're renting the rack space, power and cooling from the datacenter and you're renting the servers from the vendor - most of the leases are designed so you can refresh your hardware every 4-5 years and it's usually still cheaper than renting from a dedicated hosting company.

Once you have an established baseline for your server needs - it's almost always more capital friendly to buy the servers and keep them running for the ~5 reliable years you'll get out of them - usually break even here is 2-3 years vs renting from a provider. If you're running your servers until they fail you'll get 7-10 years out of them, provided the power cost is still worth running them (usually that is also around the 8-10 year mark depending on your power cost).

So there are many reasons you'd buy vs rent - including capital deductions and access to cheap interest rates. You can also get some pretty crazy deals (like 33% of new price) by buying 2-3 year old equipment, then continue to run them for another 4-5 years, which is the lowest cost scenario if you don't need bleeding edge.

Re: Use One Big Server (2022)

#269
post #164
post #60

Earlier quoted context omitted.

I don't know about Hetzner, but the failure case isn't usually tripping over power plugs. It's putting a longer server in the rack above/below yours and pushing the power plug out of the back of your server. Either way, stuff happens, figuring out what your actual requirements around uptime, time to response, and time to resolution is important before you build a nine nines solution when eight eights is sufficient. :…

> It's putting a longer server in the rack above/below yours and pushing the power plug out of the back of your server Are you serious? Have you ever built/operated/wired rack scale equipment? You think the power cables for your "short" server (vs the longer one being put in) are just hanging out in the back of the rack? Rack wiring has been done and done correctly for ages. Power cables on one side (if possible), da…

Yes I'm serious. My managed host took several of our machines offline when racking machines under/over ours. And they said it was because the new machines were longer and knocked out the power cables on ours.

We were their largest customer and they seemed honest even when they made mistakes that seemed silly, so we rolled our eyes and moved on with life.

Managed hosting means accepting that you can't inspect the racks and chide people for not cabling to your satisfaction. And mistakes by the managed host will impact your availability.

Re: Use One Big Server (2022)

#270
post #174

Earlier quoted context omitted.

I don’t disagree but “cores” is not a good measure of computational power.

True, but the cores on a dedicated Hetzner box obliterate the cores on an EC2 machine every time I’ve tested them. So, if anything, it understates the massive performance gap.

Hetzner also tends to have more modern SSDs with the latest nvme. Which can make a massive difference for your DB.
Post reply on HN