Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use one big server

specbranch.com

Post reply on HN
Print view View original

431–440 of 601 posts

Re: Use one big server

#431

Post by bigtunacan »

Retric wrote: post #420
bigtunacan wrote: post #417

Earlier quoted context omitted.

I can't speak for cogman10, but in my experience when you start to encounter issues of hitting the limit of "one big database" you are generally dealing with some really complicated shit and refactoring to dedicated read instances, shards, and other DB hacks are just short term solutions to buy time. The long term solutions end up being difficult to implement and can be high risk because now you have real customers (…

The other side of this is once you actually can’t scale a single DB the project has proved it’s value and you have a solid idea what you actually want. Designing let alone building something scaleable on the other hand is a great way to waste extreme effort up front when it’s completely superfluous. That’s vastly more likely to actually kill a project than some growing pains especially when most projects never scale…

You're not wrong. Probably more than 95% of applications will never outgrow one large relational database. I just think that this leads to an unfortunate, but mostly inevitable issue of complexity for the few that do hit such a level of success and scale.

Alex DeVrie (author of 'The DynamoDB Book') discusses that his approach is to essentially start all new projects with DynamoDB.

Now I don't really agree with him, yet I can't fully say he's wrong either. While we won't need it most of the time, reaching for a tool like this before we need it provides more time to really understand it when/if we reach that point.

Re: Use one big server

#433

Post by ozb »

throwfh80h82 wrote: post #406
endisneigh wrote: post #402

Earlier quoted context omitted.

Without details it's hard to really get anything from this. Could you share some?

I can share some. Had a similar experience as the parent comment. I do support "one big database" but it requires a dedicated db admin team to solve the tragedy of the commons problem. Say you have one big database. You have 300 engineers and 30-50 product managers shipping new features every day accountable to the C-Suite. They are all writing queries to retrieve the data they want. One more join, one more N+1 query…

Sure, you'll get to m6g.16xlarge; but how many companies actually have oltp requirements that exceed the limits of single servers on AWS, eg u-12tb1.112xlarge or u-24tb1.metal (that's 12-24tb memory)? I think these days the issues with high availability, cost/autoscaling/commitment, "tragedy of the commons", bureaucracy, and inter-team boundaries are much more likely to be the drawback than lack of raw power.

Re: Use one big server

#434

Post by cogman10 »

endisneigh wrote: post #402
cogman10 wrote: post #382

Earlier quoted context omitted.

> I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it. We've spent and failed at multiple multi year projects to "solve" the problem. I'm sure there are more simple problems that are easier to disentangle. But not in our case.

Without details it's hard to really get anything from this. Could you share some?

I did in the original comment.

We have over 200 monolith applications each accessing overlapping schemas of data with their own sets of stored procedures, views, and direct queries. To migrate a portion of that data out into it's own database requires, generally, refactoring a large subset of the 200 monolith apps to no longer get all the data in one query, but rather a portion of the data with the query and the rest of the data with a new service.

Sharding the data is equally difficult because even tracing who is writing the data is spread from one side of the system to the next. We've tried to do that trough an elaborate system of views, but as you can imagine, those are too slow and cover too much data for some critical applications so they end up breaking the shard. That, in and of itself, introduces additional complexity with the evolution of the products.

Couple that with the fact that even with these solutions, getting a large portion of the organization is not on board with these solutions (why can't we JUST buy more hardware? Get JUST bigger databases?) and these efforts end up being sabotaged from the beginning because not everyone thinks it's a good idea (And if you think you are different, I suggest just looking at the rest of the comments here in HN that provide 20 different solutions to the problem some of which are "why can't you just buy more hardware?")

But, to add to all of this, we also just have organizational deficiencies that have really harmed these efforts. Including things like a bunch of random scripts checked into who knows where that are apparently mission critical and reading/writing across the entire database. General for things like "the application isn't doing the right thing, so this cron job run every Wednesday will go in and fix things up" Quiet literally 1000s of those scripts have been written.

This isn't to say we've been 100% unsuccessful at splitting some of the data into it's own server. But, it's a long and hard slog.

Re: Use one big server

#435

Post by marcosdumay »

zmmmmm wrote: post #346
marcosdumay wrote: post #192

Earlier quoted context omitted.

Yeah... Dividing your work into microservices while your data is in an interdependent database doesn't lead to great results. If you are creating microservices, you must segment them all the way through.

I have to say I disagree with this ... you can only separate them if they are really, truly independent. Trying to separate things that are actually coupled will quickly take you on a path to hell. The problem here is that most of the microservice architecture divisions are going to be driven by Conway's law, not what makes any technical sense. So if you insist on separate databases per microservice, you're at high r…

Well, I stand by what I said. And you are also correct, you can only separate them if they are really truly independent. Those two are correct at the same time.

Microservices does more than encapsulation and workspace segmentation. They also distribute data locality and coherence. If you have an organizational need to break something, but not on independent parts, it's better to use some abstraction that preserves the data properties.

(In other words, microservices are almost never the answer. There are plenty of ways to organize your code, default to those other ones. And on the few cases that microservices are the answer, rest assured that you won't fail to notice it.)

Re: Use one big server

#436

Post by pottertheotter »

cogman10 wrote: post #342
cfors wrote: post #56

Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

Wonder if this would be a good use case for Tidalscale?

Re: Use one big server

#437

Post by xuki »

rszorness wrote: post #327
thayne wrote: post #265

Earlier quoted context omitted.

One issue I've seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups. I'd be interested to know if anyone has a good solution for that.

Try out pg_probackup. It works on database files directly. Restore is as fast as you can write on your ssd. I've setup a pgsql server with timescaledb recently. Continuing backup based on WAL takes seconds each hour and a complete restore takes 15 minutes for almost 300 GB of data because the 1 GBit connection to the backup server is the bottleneck.

For MySQL there is xtrabackup - https://www.percona.com/software/mysql-database/percona-xtra....

Re: Use one big server

#438

Post by [deleted] »

throwfh80h82 wrote: post #406
endisneigh wrote: post #402

Earlier quoted context omitted.

Without details it's hard to really get anything from this. Could you share some?

I can share some. Had a similar experience as the parent comment. I do support "one big database" but it requires a dedicated db admin team to solve the tragedy of the commons problem. Say you have one big database. You have 300 engineers and 30-50 product managers shipping new features every day accountable to the C-Suite. They are all writing queries to retrieve the data they want. One more join, one more N+1 query…

[deleted]

Re: Use one big server

#440

Post by krmboya »

sanderjd wrote: post #421
throwfh80h82 wrote: post #406

Earlier quoted context omitted.

I can share some. Had a similar experience as the parent comment. I do support "one big database" but it requires a dedicated db admin team to solve the tragedy of the commons problem. Say you have one big database. You have 300 engineers and 30-50 product managers shipping new features every day accountable to the C-Suite. They are all writing queries to retrieve the data they want. One more join, one more N+1 query…

I think this hits the nail right on the head, and it's the same criticism I have of and article itself: the framing is that you split up a database or use small vms or containers for performance reasons, but that's not the primary reason these things are useful; they are useful for people scaling first and foremost, and for technical scaling only secondarily. The tragedy of the commons with one big shared database is…

In uni we learnt about federated databases, i.e multiple autonomous, distributed, possibly heterogeneous databases joined together by some middleware to service user queries. I wonder how that would work for this situation, in the place of one single large database.

Federated databases are never usually mentioned in these kind of discussions involving 'web scale'. Maybe because of latency? I don't know

Post reply on HN