Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use one big server

specbranch.com

Post reply on HN
Print view View original

381–390 of 601 posts

Re: Use one big server

#381

Post by MrStonedOne »

mwcampbell wrote: post #238
MrStonedOne wrote: post #97

/tg/station, the largest open source multiplayer video game on github, gets cloudheads trying to help us "modernize" the game server for the cloud all the time. Here's how that breaks down: The servers (sorry, i mean compute) cost the same (before bandwidth, more on that at the bottom) to host one game server as we pay (amortized) per game server to host 5 game servers on a rented dedicated server. ($175/month for th…

> $175/month for the rented server with 64gb of ram and a 10gbit uplink) Wow, what provider is that?

No post body was provided.

Re: Use one big server

#382

Post by cogman10 »

makeitdouble wrote: post #380
cogman10 wrote: post #342

Earlier quoted context omitted.

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it. People often feel they should have anticipated and avoid the scaling issues altogether, but moving from a single DB to master/replica model, and/or shards or other solutions is fairly doable, and it doesn't come with worse tradeoffs than i…

> I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it.

We've spent and failed at multiple multi year projects to "solve" the problem. I'm sure there are more simple problems that are easier to disentangle. But not in our case.

Re: Use one big server

#383

Post by dsr_ »

thayne wrote: post #324
hamandcheese wrote: post #314

Earlier quoted context omitted.

If you stop at the first bullet point then you have a backup solution.

It doesn't solve the problem that sending that snapshot to a backup location takes a long time.

No, it doesn't.

It takes exactly the time that it takes, bottlenecked by:

* your disk read speed on one end and write speed on the other, modulo compression

* the network bandwidth between points A and B, modulo compression

* the size of the data you are sending

So, if you have a 10GB database that you send over a 10Gb/s link to the other side of the datacenter, it might be as little as 10 seconds. If you have a 10TB database that you send over a nominally 1GB/s link but actually there's a lot of congestion from other users, to a datacenter on the other side of the world, that might take a hundred hours or so.

rsync can help a lot here, or the ZFS differential snapshot send.

Re: Use one big server

#384

Post by fleddr »

GuB-42 wrote: post #350
fleddr wrote: post #331

Our industry summarized: Hardware engineers are pushing the absolute physical limits of getting state (memory/storage) as close as possible to compute. A monumental accomplishment as impactful as the invention of agriculture and the industrial revolution. Software engineers: let's completely undo all that engineering by moving everything apart as far as possible. Hmmm, still too fast. Let's next add virtualization an…

Actually, for those who push for these cloudy solutions, they do that in part to make data close to you. I am talking mostly about CDNs, I don't thing YouTube and Netflix would have been possible without them. Google is a US company, but you don't want people in Australia to connect to the other side of the globe every time they need to access Google services, it would be an awful waste of intercontinental bandwidth.…

You're right on the CDN part, but my criticism was highly generic, for sure this doesn't mean every single distributed architecture is a bad idea.

Re: Use one big server

#385

Post by cogman10 »

n0ts0gr33n wrote: post #368
cogman10 wrote: post #342

Earlier quoted context omitted.

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

I'd be curious to know what your company does which generates this volume of data (if you can disclose), what database you are using and how you are planning to solve this issue.

Finance. MSSQL.

There are multiple plans on how to fix this problem but they all end up boiling down to carving out domains and their owners and trying to pull apart the data from the database.

What's been keeping the lights on is "Always On" and read only replicas. New projects aren't adding load to the db and it's simply been a slow going getting stuff split apart.

What we've tried (and failed at) is sharding the data. The main issue we have is a bunch of systems reading directly from the db for common records rather than hitting other services. That means any change in structure requires a bunch of system wide updates.

Re: Use one big server

#386

Post by hot_gril »

quickthrower2 wrote: post #379

I am using Firebase on a project and I regret it. There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally. But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured. You can't structure it as nicely from a logical perspec…

Yes, it's very hard to beat Postgres IMO. You can use Firebase without using its database, and you can certainly run a service with a Postgres database without having to rent out physical servers.

Re: Use one big server

#387

Post by fleddr »

quasarj wrote: post #362
fleddr wrote: post #331

Our industry summarized: Hardware engineers are pushing the absolute physical limits of getting state (memory/storage) as close as possible to compute. A monumental accomplishment as impactful as the invention of agriculture and the industrial revolution. Software engineers: let's completely undo all that engineering by moving everything apart as far as possible. Hmmm, still too fast. Let's next add virtualization an…

Downvotes? But you're absolutely right. What an embarrassing industry to be a part of.

We're not a very serious industry. Despite uhm, it pretty much running the world. We're a joke. Sometimes I feel it doesn't even earn the term "engineering" at all, and rather than improving, it seems to get ever worse.

Which really is a stunning accomplishment in a backdrop of spectacular hardware advances, ever more educated people, and other favorable ingredients.

Re: Use one big server

#388

Post by earleybird »

icelancer wrote: post #369
maxbond wrote: post #195

Earlier quoted context omitted.

The thing that confuses me is, isn't every publicly accessible service bursty on a long timescale? Everything looks seasonal and predictable until you hit the front page of Reddit, and you don't know what day that will be. You don't decide how much traffic you get, the world does.

If you can't handle traffic from reddit or a larger site, you configured static pages and caching incorrectly, or you run your site on a Raspberry Pi, I guess.

Pi behind cloudfront/flare and you caching sorted (as you say) and you'll handle pretty much anything. (well, maybe make that a Pi-4 8gb if you can get one)

Re: Use one big server

#389

Post by cogman10 »

kedean wrote: post #348
cogman10 wrote: post #342

Earlier quoted context omitted.

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

Many databases can be distributed horizontally if you put in the extra work, would that not solve the problems you're describing? MariaDB supports at least two forms of replication (one master/replica and one multi-master), for example, and if you're willing to shell out for a MaxScale license it's a breeze to load balance it and have automatic failover.

We've done that (MSSQL Always on) and that's what keeps the lights on today. It's not, however, something that'll remain sustainable.
Post reply on HN