Live data from Hacker News

Use one big server

specbranch.com

561–570 of 601 posts

Re: Use one big server

#561

Earlier quoted context omitted.

Without details it's hard to really get anything from this. Could you share some?

I did in the original comment. We have over 200 monolith applications each accessing overlapping schemas of data with their own sets of stored procedures, views, and direct queries. To migrate a portion of that data out into it's own database requires, generally, refactoring a large subset of the 200 monolith apps to no longer get all the data in one query, but rather a portion of the data with the query and the rest…

fwiw hacking hundreds of apps literally making them worse by fragmenting their source of record doesn't sound like a good plan. it's no surprise you have saboteurs, your company probably wants to survive and your plan is to shatter its brain.

outside view: you should be trying to debottleneck your sql server if that's the plan the whole org can get behind. when they all want you to succeed you'll find a way.

Re: Use one big server

#562

Earlier quoted context omitted.

You must not have looked at MongoDB. We have been delivering fully consistent ACID transactions since 4.0 which shipped several years. Yes, Jepsen did find some issues with the initial release of ACID transactions and yes, we fixed those problems pretty rapidly.

> Yes, Jepsen did find some issues with the initial release of ACID transactions (...) By "some issues" you mean lost data and violated causal by default. https://www.infoq.com/news/2020/05/Jepsen-MongoDB-4-2-6/ And no, the issues were not fixed. Jepsen pointed out that "the newer MongoDB 4.2.6 has more problems" including “retrocausal transactions”. https://jepsen.io/analyses/mongodb-4.2.6 From Jepsen's report: > Je…

Also read the addendum from the same report:

1 Updates 2020-05-26: MongoDB identified a bug in the transaction retry mechanism which they believe was responsible for the anomalies observed in this report; a patch is scheduled for 4.2.8.

Re: Use one big server

#563
post #561

Earlier quoted context omitted.

I did in the original comment. We have over 200 monolith applications each accessing overlapping schemas of data with their own sets of stored procedures, views, and direct queries. To migrate a portion of that data out into it's own database requires, generally, refactoring a large subset of the 200 monolith apps to no longer get all the data in one query, but rather a portion of the data with the query and the rest…

fwiw hacking hundreds of apps literally making them worse by fragmenting their source of record doesn't sound like a good plan. it's no surprise you have saboteurs, your company probably wants to survive and your plan is to shatter its brain. outside view: you should be trying to debottleneck your sql server if that's the plan the whole org can get behind. when they all want you to succeed you'll find a way.

> fwiw hacking hundreds of apps literally making them worse by fragmenting their source of record doesn't sound like a good plan. it's no surprise you have saboteurs, your company probably wants to survive and your plan is to shatter its brain.

The brain is already shattered. This wouldn't "literally make them worse", instead it would say that "now instead of everyone in the world hitting the users table directly and adding or removing data from that table, we have one service in charge of managing users".

Far too often we have queries like

SELECT b.*, u.username FROM Bar b JOIN users u ON b.userId = u.id

And why is this query doing that? To get a human readable username that isn't needed but at one point years ago made it nicer to debug the application.

> you should be trying to debottleneck your sql server if that's the plan the whole org can get behind.

Did you read my post? We absolutely HAVE been working, for years now, at "debottlenecking our sql server". We have a fairly large team of DBAs (about 30) who's whole job is "debottlenecking our sql server". What I'm saying is that we are, and have been, at the edge (and more often than not over the edge) of tipping over. We CAN'T buy our way out of this with new hardware because we already have the best available hardware. We already have read only replicas. We already have tried (and failed at) sharding the data.

The problem is data doesn't have stewards. As a result, we've spent years developing application code where nobody got in the way of saying "Maybe you shouldn't join these two domains together? Maybe there's another way to do this?"

Re: Use one big server

#564

/tg/station, the largest open source multiplayer video game on github, gets cloudheads trying to help us "modernize" the game server for the cloud all the time. Here's how that breaks down: The servers (sorry, i mean compute) cost the same (before bandwidth, more on that at the bottom) to host one game server as we pay (amortized) per game server to host 5 game servers on a rented dedicated server. ($175/month for th…

well of course, you can't scale SS13 servers, cloud is for stuff that scales in parallel like backends

Re: Use one big server

#565
post #452

Earlier quoted context omitted.

I've never understood this logic for webapps. If you're building a web application, congratulations, you're building a distributed system, you don't get a choice. You can't actually use transactional integrity or ACID compliance because you've got to send everything to and from your users via HTTP request/response. So you end up paying all the performance, scalability, flexibility, and especially reliability costs of…

Http requests work great with relational dbs. This is not UDP. If the TCP connection is broken, an operation will either have finished or stopped and rolledback atomically and unless you've placed unneeded queues in there, you should know of success immediately. When you get the http response, you will know the data is fully committed, data that uses it can be refreshed immediately and is accessible to all other syst…

But you can't do interaction between the two ends of a HTTP request. The caller makes an inert request, whatever processing happens downstream of that might as well be offline because it's not and can never be interactive within a single transaction.

Re: Use one big server

#566
post #515

Earlier quoted context omitted.

> I really don't understand how anything of what you wrote follows from the fact that you're building a web-app. Why do you lose user data when two users do anything at the same time? That has never happened to me with any RDBMS. > And why would HTTP requests prevent me from using transactional logic? If a user issues a command such as "copy this data (a forum thread, or a Confluence page, or whatever) to a different…

> Sure, if you can represent what the user wants to do as a "command" like that, that doesn't rely on a particular state of the world, then you're fine. Note that this is also exactly the case that an eventually consistent event-sourcing style system will handle fine. Yes, but the event-sourcing system (or similar variants, such as CRDTs) is much more complex. It's true that it buys you some things (like the ability…

> Yes, but the event-sourcing system (or similar variants, such as CRDTs) is much more complex.

It's really not. An RDBMS usually contains all of the same stuff underneath the hood (MVCC etc.), it just tries to paper over it and present the illusion of a single consistent state of the world, and unfortunately that ends up being leaky.

> a) that's simply not a problem in all situations. People will generally not update their user profile concurrently with other users, for example. So it only applies to situations where data is truly shared across multiple users,

Sure - but those situations are ipso facto situations where you have no need for transactions.

> b) the problem of users overwriting other users' data is inherent to the problem domain; you will, in the end, have to decide which version is the most recent regardless of which technology you use. The one thing that evens etc. buy you is a version history (which btw can also be implemented with a RDBMS), but if you want to expose that in the UI so the user can go back, you have to do additional work anyway - it doesn't come for free.

True, but what does come for free is thinking about it when you're designing your dataflow. Using an event sourcing style forces you to confront the idea that you're going to have concurrent updates going on, early enough in the process that you naturally design your data model to handle it, rather than imagining that you can always see "the" current state of the world.

> c) Meanwhile, the RDBMS will at least guarantee that the data is always in a consistent state. Users overwriting other users' data is unfortunate, but corrupted data is worse.

I'm not convinced, because the way it accomplishes that is by dropping "corrupt" data on the floor. If user A tries to save new post B in thread C, but at the same time user D has deleted that thread, then in a RDBMS where you're using a foreign key the only thing you can do is error and never save the content of post B. In an event sourcing system you still have to deal with the fact that the post belongs in a nonexistent thread eventually, but you don't start by losing the user's data, and it's very natural to do something like mark it as an orphaned post that the user can still see in their own post history, which is probably what you want. (Of course you can achieve that in the RDBMS approach, but it tends to involve more complex logic, giving up on foreign keys and accepting tha you have to solve the same data integrity problems as a non-ACID system, or both).

> d) You can solve the "concurrent modification" issue in a variety of ways, depending on the frequency of the problem, without having to implement a complex event-sourced system. For example, a lock mechanism is fairly easy to implement and useful in many cases. You could also, for example, hash the contents of what the user is seeing and reject the change if there is a mismatch with the current state (I've never tried it, but it should work in theory).

That sounds a whole lot more complex than just sticking it an event sourcing system. Especially when the problem is rare, it's much better to find a solution where the correct behaviour naturally arises in that case, than implement some kind of ad-hoc special case workaround that will never be tested as rigorously as your "happy path" case.

Re: Use one big server

#567
post #528

Earlier quoted context omitted.

None of that means anything. The web is slower than ever. Desktop apps 20 years ago were faster than today's garbage. We failed.

> None of that means anything. What are you having difficulty understanding? I'll be happy to try help. > The web is slower than ever. No it isn't. > Desktop apps 20 years ago were faster than today's garbage. Some are, some aren't. For the same thing they clearly aren't. A typewriter makes your PC of 20 years ago look glacial garbage, if that's your standard. > We failed. Speak for yourself. Computers are used far m…

I must have hit a nerve. Saying that computers are used more often and by more people makes my assessment even more impactful.

Re: Use one big server

#568
One big sever, one big program, and one big 10x developer. Deploy websphere when you need isolation. The industry truly is going in spiral. Although, I must admit cloud providers really overplayed their hand when it comes to performance/buck and complexity.

Re: Use one big server

#569
post #478
post #446

Earlier quoted context omitted.

> Also smaller units being released means fewer changes and likely fewer failed releases. The interfaces are the hard part, so you may have fewer internal failures but problems between services seem more likely.

Yeah, not to mention all the extra operational issues and failure modes that come with RPCs vs function calls. Integration testing and release coordination both become more difficult as well. But hundreds of people contributing to a single binary is probably not realistic; at some point you'll need to factor it into pieces that can be can have somewhat independent operations.

I’m not sure one binary is the scaling problem you’d hit first though - makes me wonder just how many people can work on something that gets compiled together into a monolith

Re: Use one big server

#570
post #346

Earlier quoted context omitted.

I have to say I disagree with this ... you can only separate them if they are really, truly independent. Trying to separate things that are actually coupled will quickly take you on a path to hell. The problem here is that most of the microservice architecture divisions are going to be driven by Conway's law, not what makes any technical sense. So if you insist on separate databases per microservice, you're at high r…

>> If you are creating microservices, you must segment them all the way through. > I have to say I disagree with this ... you can only separate them if they are really, truly independent. Trying to separate things that are actually coupled will quickly take you on a path to hell. I could be misinterpreting both you and GP, but sounds like you agree with GP - if you can't segment them all the way through, maybe they s…

Perhaps - but I think they are underestimating the organisational reasons to separate services from each other. If you are really going to say "we can't separate any two things that have any shared persistent data" then you may just end up with a monolith and all the problems that come from that (gridlock because every team needs to agree before it can be updated / released etc).
Post reply on HN