Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use one big server

specbranch.com

Post reply on HN
Print view View original

301–310 of 601 posts

Re: Use one big server

#301

Post by jmull »

galkk wrote: post #274

One of first experiences in my professional career was situation when "one big server" that was serving the system that was making money actually failed on Friday, HP's warranty was like next or 2 business days to get a replacement. The entire situation ended up having conference call with multiple department directors who were deciding which server from other systems to cannibalize (even if it is underpowered) to ge…

The article is really talking about one big server plus a backup vs. cloud providers.

Re: Use one big server

#303

Post by nick__m »

thayne wrote: post #265
belak wrote: post #64

Earlier quoted context omitted.

This is absolutely true - when I was at Bitbucket (ages ago at this point) and we were having issues with our DB server (mostly due to scaling), almost everyone we talked to said "buy a bigger box until you can't any more" because of how complex (and indirectly expensive) the alternatives are - sharding and microservices both have a ton more failure points than a single large box. I'm sure they eventually moved off t…

One issue I've seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups. I'd be interested to know if anyone has a good solution for that.

On mariadb you can tell the replica to enter into a snapshotable state[1] and take a simple lvm snapshot, tell the the database it's over, backup your snapshot somewhere else and finally delete the snapshot.

1) https://mariadb.com/kb/en/storage-snapshots-and-backup-stage...

Re: Use one big server

#304

Post by fleddr »

1500100900 wrote: post #165
bartread wrote: post #151

Earlier quoted context omitted.

Not to mention, backups, restores, and disaster recovery are so much easier with One Big Database™.

How is backup restoration any easier if your whole PostgreSQL cluster goes back in time when you only wanted to rewind that one tenant?

Your scenario is data recovery, not backup restoration. Wildly different things.

Re: Use one big server

#305

Post by chromatin »

ryanisnan wrote: post #74
cfors wrote: post #56

Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…

Definitely use a big database, until you can't. My advice to anyone starting with a relational data store is to use a proxy from day 1 (or some point before adding something like that becomes scary). When you need to start sharding your database, having a proxy is like having a super power.

Are there postgres proxies that can specifically facilitate sharding / partitioning later?

Re: Use one big server

#306

Post by Gigachad »

manigandham wrote: post #200
Closi wrote: post #61

Earlier quoted context omitted.

Breaking apart a stateless microservice and then basing it around a giant single monolithic database is pretty pointless - at that stage you might as well just build a monolith and get on with it as every microservice is tightly coupled to the db.

Why would you break apart a microservice? Any why do you need to use/split into microservices anyway? 99% of apps are best fit as monolithic apps and databases and should focus on business value rather than scale they'll never see.

Where I work we are looking at it because we are starting to exceed the capabilities of one big database. Several tables are reaching the billions of rows mark and just plain inserts are starting to become too much.

Re: Use one big server

#307

Post by adgjlsfhk1 »

Closi wrote: post #204
manigandham wrote: post #200

Earlier quoted context omitted.

Why would you break apart a microservice? Any why do you need to use/split into microservices anyway? 99% of apps are best fit as monolithic apps and databases and should focus on business value rather than scale they'll never see.

Totally agree. I guess I just don't see the value in having a monolith made up of microservices - you might as well just build a monolith if you are going down that route. And if your application fits the microservices pattern better, then you might as well go down the microservices pattern properly and not give them a big central DB.

The one advantage of microservice on a single database model is that it lets you test the independent components much more easily while avoiding the complexity of database sharding.

Re: Use one big server

#308

Post by threeseed »

Guid_NewGuid wrote: post #101
cfors wrote: post #56

Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…

I think a strong test a lot of "let's use Google scale architecture for our MVP" advocates fail is: can your architecture support a performant paginated list with dynamic sort, filter and search where eventual consistency isn't acceptable? Pretty much every CRUD app needs this at some point and if every join needs a network call your app is going to suck to use and suck to develop.

> if every join needs a network call your app is going to suck to use and suck to develop.

And yet developers do this every single day without any issue.

It is bad practice to have your authentication database be the same as your app database. Or you have data coming from SaaS products, third party APIs or a cloud service. Or even simply another service in your stack. And with complex schemas often it's far easier to do that join in your application layer.

All of these require a network call and join.

Re: Use one big server

#309

Post by vlovich123 »

PeterCorless wrote: post #125

We have a different take on running "one big database." At ScyllaDB we prefer vertical scaling because you get better utilization of all your vCPUs, but we still will keep a replication factor of 3 to ensure that you can maintain [at least] quorum reads and writes. So we would likely recommend running 3x big servers. For those who want to plan for failure, though, they might prefer to have 6x medium servers, because…

> At that point, you are updating your resume, and checking on the quality of your parachute

The ordering of these events seems off but that's understandable considering we're talking about distributed systems.

Re: Use one big server

#310

Post by hamandcheese »

dsr_ wrote: post #282
thayne wrote: post #265

Earlier quoted context omitted.

One issue I've seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups. I'd be interested to know if anyone has a good solution for that.

Here's the way it works for, say, Postgresql: - you rsync or zfs send the database files from machine A to machine B. You would like the database to be off during this process, which will make it consistent. The big advantage of ZFS is that you can stop PG, snapshot the filesystem, and turn PG on again immediately, then send the snapshot. Machine B is now a cold backup replica of A. Your loss potential is limited to…

Do you even have to stop Postgres if using ZFS snapshots? ZFS snapshots are atomic, so I’d expect that to be fine. If it wasn’t fine, that would also mean Postgres couldn’t handle power failure or other sudden failures.
Post reply on HN