Live data from Hacker News

Use one big server

specbranch.com

371–380 of 601 posts

Re: Use one big server

#371

Earlier quoted context omitted.

I'm glad this is becoming conventional wisdom. I used to argue this in these pages a few years ago and would get downvoted below the posts telling people to split everything into microservices separated by queues (although I suppose it's making me lose my competitive advantage when everyone else is building lean and mean infrastructure too). In my mind, reasons involve keeping transactional integrity, ACID compliance…

'over the wire' is less obvious than it used to be. If you're in k8s pod, those calls are really kernel calls. Sure you're serializing and process switching where you could be just making a method call, but we had to do something. I'm seeing less 'balls of mud' with microservices. Thats not zero balls of mud. But its not a given for almost every code base I wander into.

To clarify, I think stateless microservices are good. It's when you have too many DBs (and sometimes too many queues) that you run into problems.

A single instance of PostgreSQL is, in most situations, almost miraculously effective at coordinating concurrent and parallel state mutations. To me that's one of the most important characteristic of an RDBMS. Storing data is a simpler secondary problem. Managing concurrency is the hard problem that I need most help with from my DB and having a monolithic DB enables the coordination of everything else including stateless peripheral services without resulting in race conditions, conflicts or data corruption.

SQL is the most popular mostly functional language. This might be because managing persistent state and keeping data organized and low entropy, is where you get the most benefit from using a functional approach that doesn't add more state. This adds to the effectiveness of using a single transactional DB.

I must admit that even distributed DBs, like Cockroach and Yugabyte have recognized this and use the PostgreSQL syntax and protocol. This is good though, it means that if you really need to scale beyond PostgreSQL, you have PostgreSQL compatible options.

Re: Use one big server

#372

Earlier quoted context omitted.

I think a strong test a lot of "let's use Google scale architecture for our MVP" advocates fail is: can your architecture support a performant paginated list with dynamic sort, filter and search where eventual consistency isn't acceptable? Pretty much every CRUD app needs this at some point and if every join needs a network call your app is going to suck to use and suck to develop.

> if every join needs a network call your app is going to suck to use and suck to develop. And yet developers do this every single day without any issue. It is bad practice to have your authentication database be the same as your app database. Or you have data coming from SaaS products, third party APIs or a cloud service. Or even simply another service in your stack. And with complex schemas often it's far easier to…

> And yet developers do this every single day without any issue.

And users suffer through unresponsive interfaces and long load times every single day...

Re: Use one big server

#373
post #317

Earlier quoted context omitted.

I don't believe you. Eventual consistency is how the real world works, what possible use case is there where it wouldn't be acceptable? Even if you somehow made the display widget part of the database, you can't make the reader's eyeballs ACID-compliant.

Yeah, I can attest that even banks are really using best effort eventual consistency. However, I think it is very difficult to reason about with systems that try to use eventual consistency as an abstraction. It's a lot easier to think about explicitly when you have one data source/event that propagates outwards through systems with stronger individual guarantees than eventual consistency.

IMO having event streams as first class is the best way to think about things. Then you don't need particularly strong guarantees downstream - think something like Kafka where the only guarantee is that events for the same key will always be processed in order, and it turns out that that's enough to build a system with clear, reliable behaviour that you can reason about quite easily.

Re: Use one big server

#374

Earlier quoted context omitted.

Why would you break apart a microservice? Any why do you need to use/split into microservices anyway? 99% of apps are best fit as monolithic apps and databases and should focus on business value rather than scale they'll never see.

> 99% of apps are best fit as monolithic apps and databases and should focus on business value rather than scale they'll never see You incorrectly assume that 99% of apps are building these architectures for scalability reasons. When in reality it's far more for development productivity, security, use of third party services, different languages etc.

The only microservices architecture website I've ever dealt with has been an absolute nightmare compared to what the same monolith should have been.

Re: Use one big server

#376
post #282

Earlier quoted context omitted.

Here's the way it works for, say, Postgresql: - you rsync or zfs send the database files from machine A to machine B. You would like the database to be off during this process, which will make it consistent. The big advantage of ZFS is that you can stop PG, snapshot the filesystem, and turn PG on again immediately, then send the snapshot. Machine B is now a cold backup replica of A. Your loss potential is limited to…

This isn't really a backup, it's redundancy which is good thing but not the same as a backup solution. You can't get out of a drop table production type event this way.

If you add a delay of say 30 minutes for one of your replicas, you have another option in a "drop table" type event.

Re: Use one big server

#377
post #324

Earlier quoted context omitted.

If you stop at the first bullet point then you have a backup solution.

It doesn't solve the problem that sending that snapshot to a backup location takes a long time.

Unless your storage is already mirrored off-site. Ex: EMC srdf

Re: Use one big server

#378
post #348

Earlier quoted context omitted.

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

Many databases can be distributed horizontally if you put in the extra work, would that not solve the problems you're describing? MariaDB supports at least two forms of replication (one master/replica and one multi-master), for example, and if you're willing to shell out for a MaxScale license it's a breeze to load balance it and have automatic failover.

I worked at a mobile game company for years and years, and our #1 biggest scaling concern was DB write throughput. We used Percona's MySQL fork/patch/whatever, we tuned as best we could, but when it comes down to it, gaming is a write-heavy application rather than the read-heavy applications I'm used to from ecommerce etc.

Sharding things out and replicating worked for us, but only because we were microservices-y and we were able to split our schemas up between different services. Still, there was one service that required the most disk space, the most write throughput, the most everything.

(IIRC it was the 'property' service, which recorded everything anyone owned in our games and was updated every time someone gained, lost, or used any item, building, ally, etc).

We did have two read replicas and the service didn't do reads from the primary so that it could focus on writes, but it was still a heavy load that was only solved by adding hardware, improving disks, adding RAM, and so on.

Re: Use one big server

#379
I am using Firebase on a project and I regret it.

There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally.

But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured.

You can't structure it as nicely from a logical perspective compared to a relational DB. Because you can't join without pulling data from all over the place. Because the data isn't in one place. It is hard to do joins both in terms of performance and in terms of developer ergonomics.

I really miss SELECT A.X, B.Y FROM A JOIN B ON A.ID = B.AID; when using Firebase.

You have to make data storage decisions early on, and it is hard to change you mind later. It is hard to migrate (and may be expensive if you have a lot of existing data).

I picked Firebase for the wrong reason (I thought it would make MVP quicker to set up). But the conveniences it provides are outweighed by having to structure your data for distribution across servers.

Instead next time I would go relational, then when I hit a problem do that bit distributed. Most tables have 1000s of records. Maybe millions. The table with billions might need to go out to something distributed.

Market gap??:

Let me rent real servers, but expose it in a "serverless" "cloud-like" way, so I don't have to upgrade the OS and all that kind of stuff.

Re: Use one big server

#380
post #56

Yep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want t…

> Use One Big Database. > Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). This works until it doesn't and then you land in the position…

I don't know what's the complexity of your project, but more often than not the feeling of doom coming from hitting that wall is bigger than the actual effort it takes to solve it.

People often feel they should have anticipated and avoid the scaling issues altogether, but moving from a single DB to master/replica model, and/or shards or other solutions is fairly doable, and it doesn't come with worse tradeoffs than if you sharded/split services from the start. It always feels fragile and bolt on compared to the elegance of the single DB, but you'd also have many dirty hacks to have a multi DB setup work properly.

Also, you do that from a position where you usually have money, resources and a good knowledge of your core parts, which is not true when you're still growing full speed.

Post reply on HN