Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use one big server

specbranch.com

Post reply on HN
Print view View original

411–420 of 601 posts

Re: Use one big server

#411

Post by adra »

pphysch wrote: post #372
threeseed wrote: post #308

Earlier quoted context omitted.

> if every join needs a network call your app is going to suck to use and suck to develop. And yet developers do this every single day without any issue. It is bad practice to have your authentication database be the same as your app database. Or you have data coming from SaaS products, third party APIs or a cloud service. Or even simply another service in your stack. And with complex schemas often it's far easier to…

> And yet developers do this every single day without any issue. And users suffer through unresponsive interfaces and long load times every single day...

Most of our cloud hosted request/responses are within the realm of 1-10ms, and that's with the actual request being processed on the other side. Unless there's a poorly performing O(N) stinker in the works, most requests can be served with most latency being recorded user->datacenter, not machine to machine overhead. This article is a lot bonkers.

Re: Use one big server

#412

Post by endisneigh »

throwfh80h82 wrote: post #406
endisneigh wrote: post #402

Earlier quoted context omitted.

Without details it's hard to really get anything from this. Could you share some?

I can share some. Had a similar experience as the parent comment. I do support "one big database" but it requires a dedicated db admin team to solve the tragedy of the commons problem. Say you have one big database. You have 300 engineers and 30-50 product managers shipping new features every day accountable to the C-Suite. They are all writing queries to retrieve the data they want. One more join, one more N+1 query…

I do not disagree at all that what you are describing can happen. What I'm not understanding is why they're failing at multi year attempts to fix this.

Even in your scenario you could identify schemas and tables that can be separated and moved into a different database or at maturity into a more scalable NoSQL variety.

Generally once you get to the point that is being described that means you have a very strong sense on the of queries you are making. Once you have that it's not strictly necessary to even use a RDBMS, or at the very least, a single database server.

Re: Use one big server

#413

Post by [deleted] »

Twisell wrote: post #298
mgiampapa wrote: post #291

Earlier quoted context omitted.

This isn't really a backup, it's redundancy which is good thing but not the same as a backup solution. You can't get out of a drop table production type event this way.

The previous commenter was probably unaware of the various way to backup recent postgresql release. For what you describe a "point in time recovery" backup would probably be the more adequate flavor https://www.postgresql.org/docs/current/continuous-archiving... It was first release around 2010 and gained robustness with every release hence not everyone is aware of it. The for instance I don't think it's really requi…

[deleted]

Re: Use one big server

#414

Post by sagarm »

alexpotato wrote: post #262

My favorite summary of why not to use microservices is from Grug: "grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too seem very confusing to grug" https://grugbrain.dev/#grug-on-microservices

IMO microservices primarily solve organizational problems, not technical problems.

They allow a team to release independently of other teams that have or want to make different risk/velocity tradeoffs. Also smaller units being released means fewer changes and likely fewer failed releases.

Re: Use one big server

#415

Post by wiremine »

I see these debates and wish there was an approach that scaled better.

A single server (and a backup) really _is_ great. Until it's not, for whatever reason.

We need more frameworks that scale from a single box to many boxes, without starting over from scratch. There are a lot of solid approaches: Erlang/Elxir and the actor model comes to mind. But that approach is not perfect, and it's far from common place.

Re: Use one big server

#416

Post by chx »

Yeah, I've been saying this for a long long time now, an early blog post of mine http://drupal4hu.com/node/305.html and this madness just got worse because of Kubernetes et al. Kubernetes is a Google solution. Are you sure Google-sized solutions are right for your organization?

Also, an equally pseudo controversial viewpoint: it's almost always cheaper to be down than engineering a HA architecture. Take a realistic look at downtime causes outside of your control -- for example, your DDoS shield provider going down etc. etc. and then consider how much downtime a hardware failure adds and now think. Maybe a manual failover master-slave is enough or perhaps even that's overkill? How much money does the business lose by being down versus how much it costs to protect from it? And can you really protect from it? Are you going to have regular drills to practice the failover -- and absurdly, will the inevitable downtime from failing a few of those be larger than a single server downtime? I rarely see posts about weighing these while the general advice of avoiding single points of failure -- which is very hard -- is abundant.

Re: Use one big server

#417

Post by bigtunacan »

makeitdouble wrote: post #380
cogman10 wrote: post #342

Earlier quoted context omitted.

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it. People often feel they should have anticipated and avoid the scaling issues altogether, but moving from a single DB to master/replica model, and/or shards or other solutions is fairly doable, and it doesn't come with worse tradeoffs than i…

I can't speak for cogman10, but in my experience when you start to encounter issues of hitting the limit of "one big database" you are generally dealing with some really complicated shit and refactoring to dedicated read instances, shards, and other DB hacks are just short term solutions to buy time.

The long term solutions end up being difficult to implement and can be high risk because now you have real customers (maybe not so happy because now slow db) and probably not much in house experience for dealing with such large scale data; and an absolute lack of ability to hire existing talent as the few people that really can solve for it are up to their ears in job offers.

Re: Use one big server

#418

Post by andsoitis »

"In total, this server has 128 cores with 256 simultaneous threads. With all of the cores working together, this server is capable of 4 TFLOPs of peak double precision computing performance. This server would sit at the top of the top500 supercomputer list in early 2000. It would take until 2007 for this server to leave the top500 list. Each CPU core is substantially more powerful than a single core from 10 years ago, and boasts a much wider computation pipeline."

Re: Use one big server

#419

Post by langsoul-com »

So much of the latest tech news and solutions are from huge companies.

Let's be real here, how many of us get 10-100 million users/requests etc? My blog, langsoul.com, has 2, myself and a bot .

Simple dumb solutions seems best for 99% of cases, then, if you ever hit that 1%, well, you'll have shit tons of money to deal with it then!

Re: Use one big server

#420

Post by Retric »

bigtunacan wrote: post #417
makeitdouble wrote: post #380

Earlier quoted context omitted.

I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it. People often feel they should have anticipated and avoid the scaling issues altogether, but moving from a single DB to master/replica model, and/or shards or other solutions is fairly doable, and it doesn't come with worse tradeoffs than i…

I can't speak for cogman10, but in my experience when you start to encounter issues of hitting the limit of "one big database" you are generally dealing with some really complicated shit and refactoring to dedicated read instances, shards, and other DB hacks are just short term solutions to buy time. The long term solutions end up being difficult to implement and can be high risk because now you have real customers (…

The other side of this is once you actually can’t scale a single DB the project has proved it’s value and you have a solid idea what you actually want.

Designing let alone building something scaleable on the other hand is a great way to waste extreme effort up front when it’s completely superfluous. That’s vastly more likely to actually kill a project than some growing pains especially when most projects never scale past a single reasonably optimized database.

Post reply on HN