Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use One Big Server (2022)

specbranch.com

Post reply on HN
Print view View original

321–330 of 330 posts

Re: Use One Big Server (2022)

#321

Post by throwaway7783 »

runako wrote: post #260
throwaway7783 wrote: post #131

Earlier quoted context omitted.

Probably not for $200/mo EC2, but AWS/GCP in general * Centralized logging, log search, log based alerting * Secrets manager * Managed kubernetes * Object store * Managed load balancers * Database HA * Cache solutions ... Can I run all these by myself? Sure. But I'm not in this business. I just want to write software and run that. And yes, I have needed most of this from day 1 for my startup. For a personal toy proje…

> Centralized logging, log search, log based alerting Do people really use the bare CloudWatch logs as an answer for log search? I find it terrible and pretty much always recommend something like DataDog or Splunk or New Relic.

we are on GCP. Logs Explorer on GCP is pretty good

Re: Use One Big Server (2022)

#322

Post by toast0 »

kapone wrote: post #311
toast0 wrote: post #269

Earlier quoted context omitted.

Yes I'm serious. My managed host took several of our machines offline when racking machines under/over ours. And they said it was because the new machines were longer and knocked out the power cables on ours. We were their largest customer and they seemed honest even when they made mistakes that seemed silly, so we rolled our eyes and moved on with life. Managed hosting means accepting that you can't inspect the rack…

I hope that "managed host" got fired in a heartbeat and you moved elsewhere. Because they don't know WTF they're doing. As simple as that.

We did eventually move elsewhere because of acquisition. Of course those guys didn't even bother to run LACP and so our systems would regularly go offline for a bit whenever someone wanted to update a switch. I was a lot happier at the host that sometimes bumped the power cables.

Firing a host where you've got thousands of servers is easier said than done. We did do a quote exercise with another provider that could have supported us, and it didn't end up very competitive ... and it wouldn't have been worth the transition. Overall, there were some derpy moments, but I don't think we would have been happier anywhere else, and we didn't want to rent cages and run our own servers.

Re: Use One Big Server (2022)

#323

Post by throwaway7783 »

troupo wrote: post #320
throwaway7783 wrote: post #318

Earlier quoted context omitted.

.. who are okay with services going down here and there

Which is the vast majority of services.

Okay. If the premise is "you don't have to worry about downtimes and only need to serve a few hundred thousand users and no data intensive use cases", then I guess you can do whatever and it'll still be okay.

Re: Use One Big Server (2022)

#324

Post by throwaway7783 »

Spooky23 wrote: post #237
eska wrote: post #147

Earlier quoted context omitted.

Now imagine your solution is not on a distributed system and go through that list. Centralized logging? There is nothing to centralized. Secrets management? There are no secrets to be constantly distributed to various machines on a network. Load balancing? In practice most people for most work don’t use it because of actually outgrowing hardware, but because they have to provision to shared hardware without exclusivi…

It sounds like you make a living doing stuff that has an incredibly small, ninja-like team, has a very low change rate, or is something that nobody really cares about. Things like RPO/RTO, multi-tenancy, logging, etc don't matter. That's amazing. I wish I could do the same. Unfortunately, I cannot run my business on a single server in a cage somewhere for a multitude of reasons. So I use AWS, a couple of colos and Sa…

Exactly. I don't quite understand how people say you just need a box. It certainly is much higher performant than a cloud VM, but that is not the only thing there is to run a software well. It all adds up bit by bit. It surely seems to be the way to go at some scale (or no customers who care).

Re: Use One Big Server (2022)

#325

Post by gethly »

HankStallone wrote: post #319
benjiro wrote: post #214

Earlier quoted context omitted.

> I will definitely buy my own hardware and collocate. Even colocation is often fraud with issues. I shall not mentioned the plectra of dead hardware from datacenter electricity failures. Ironically, my home has more stable electricity then some datacenters lol . Unless you running a business where a few minutes downtime will cost you millions, most companies can literally run their own servers from their basements.…

Sometimes I wonder why I'm not running my servers from my home, considering my 1Gb fiber has 3ms latency, and a good UPS would get me through all but a couple of the longest power outages I've had in the last 15 years. As long as I'm hosting small business web sites or something like that, and not critical banking or hospital systems, there's no reason it wouldn't be fine. When my colo contract runs out in a couple y…

In the past, ISPs were forbidding this for home connections. But nowadays it looks like it should not be an issue for basic websites.

Re: Use One Big Server (2022)

#326

Post by rcxdude »

throwaway7783 wrote: post #314
ozozozd wrote: post #141

Earlier quoted context omitted.

You need database HA and load balancers on day 1? You must be doing truly a lot of growth prior to building. Or perhaps insisting on tiny VMs for your loads?

HA is for availability. I don't want downtime for my enterprise customers. Are your customers okay with downtime? And as soon as you have more than one nodes, you need some kind of a load balancer in the front.

In practice, until you're at a certain scale, software bugs are more of a threat to your availability than hardware failures or maintenance downtime, and the cloud does nothing for you there (in fact, the additional complexity is likely to make it worse). Modern hardware is pretty reliable, more so than a given ec2 instance, for example.

Re: Use One Big Server (2022)

#327

Post by vidarh »

Havoc wrote: post #219
vidarh wrote: post #109

Earlier quoted context omitted.

But they generally don't. Most people don't have large enough daily fluctuations for these demand curves to flatten out enough. And the providers also need enough capacity to handle unforeseen spikes. Which is also why none of them will let you scale however far you want - they still impose limits so they can plan the excess they need.

> And the providers also need enough capacity to handle unforeseen spikes. Indeed but the headroom the cloud needs overall is less than every customers individual worst case scenarios added up. They’d take a percentage of that total because statistically a situation where 100% of customers are at 100% of their peak at 100% same point in time is improbable Must admit little surprised this logic isn’t self evident

The logic isn't self evident because it is irrelevant that their total demand doesn't add up this way, because the unit cost of the capacity is higher, and so the cost still ends up being far higher.

The unit cost is higher for many reasons, but the two basic ones are margins (exorbitant ones; this is not an efficient market) and that the providers also need to charge for the unused capacity to meet demand from customers behaving in ways they don't on fixed capacity systems, where spreading workloads over time wherever possible tends to become the norm.

The demand curves when you're charged for total demand over time rather than peak demand are fundamentally different, and so while you're right the peaks rarely add up in the worst possible way, empirically the peaks end up high enough that cloud compute is expensive even before the exorbitant margins the large cloud providers charge.

Re: Use One Big Server (2022)

#328

Post by vidarh »

SatvikBeri wrote: post #253
vidarh wrote: post #108

Earlier quoted context omitted.

This is usually only true of you lift and shift your AWS setup exactly as-is, instead of looking at what hardware will run your setup most efficiently. The biggest cost with AWS also isn't compute, but egress - for bandwidth heavy setups you can sometimes finance the entirety of the servers from a fraction of the savings in egress. I cost optimize setups with guaranteed caps at a proportion of savings a lot of the ti…

I'd definitely be curious to hear how you'd approach our overall situation. We don't have significant egress costs, nor has any place I've worked with before. Our AWS costs are about 80% EC2 and Fargate, with the rest scattered over various services. Roughly half our spend is on 24/7 reserved instances, while the other half is in bursty analytics workloads. Our workloads are primarily memory-bound, and AWS offers pre…

I don't know what to say to this. X2gd instances are horrifically expensive - if you haven't trivially found cheaper machines elsewhere, there are essential details of your requirements you're leaving out.

Re: Use One Big Server (2022)

#329

Post by throwaway7783 »

rcxdude wrote: post #326
throwaway7783 wrote: post #314

Earlier quoted context omitted.

HA is for availability. I don't want downtime for my enterprise customers. Are your customers okay with downtime? And as soon as you have more than one nodes, you need some kind of a load balancer in the front.

In practice, until you're at a certain scale, software bugs are more of a threat to your availability than hardware failures or maintenance downtime, and the cloud does nothing for you there (in fact, the additional complexity is likely to make it worse). Modern hardware is pretty reliable, more so than a given ec2 instance, for example.

Often, software bugs cause issues with machines (uncontrolled logging overwhelming disk space and killing everything, instead of objectstore absorbing it, or a memory bug that kills the process over a few days and needing manual or custom monitoring scripts instead of k8s handling this until its root caused, and so on)

Re: Use One Big Server (2022)

#330

Post by aflukasz »

Biganon wrote: post #285
railorsi wrote: post #63

Earlier quoted context omitted.

What’s the issue with running Postgres inside a docker container + regular backups? Never had problem and relatively easy to manage.

Why use a docker container? I run Postgres as is, what would I gain with running it in a container?

You can decouple Postgres and surrounding userspace upgrade cycles from your host os, if this is something that you want. Or run multiple different PG versions (have independent upgrades schedule) without being tied to the host os specific mechanisms for that.
Post reply on HN