Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good system design

seangoedecke.com

Post reply on HN
Print view View original

361–370 of 400 posts

Re: Good system design

#361

Post by avinassh »

OP has this first:

> If a system has distributed-consensus mechanisms, many different forms of event-driven communication, CQRS, and other clever tricks, I wonder if there’s some fundamental bad decision that’s being compensated for (or if the system is just straightforwardly over-designed).

then later down in the article:

> Send as many read queries as you can to database replicas. A typical database setup will have one write node and a bunch of read-replicas. The more you can avoid reading from the write node, the better - that write node is already busy enough doing all the writes.

isn't this same as CQRS

Re: Good system design

#362

Post by mattmanser »

Yokohiii wrote: post #114
mattmanser wrote: post #51

Earlier quoted context omitted.

Most ORMs will happily let you map stored procedures and views to a class, you can have as many models as you want. So your point doesn't really make sense. The author's said nothing about ORMs. It feels like you're trying to post a personal beef about ORMs that's entirely against the "pragmatic" software design engineering the author's opining. Using ORMs to massively reduce your boiler-plate CRUD code, then using r…

The way you describe it, it would be ideal if ORMs would only handle very basic CRUD and force you to use raw sql for complex queries. But that's not reality and not how they are used, not always. In my opinion some devs take pride to do everything with their favorite ORM. I think if an app uses 90% ORM code with the remains as raw queries, a junior is inclined to favor ORM code and is also less exposed to actually w…

And the ORM free code has massive downsides, not limited to if you add/change a column code can break at runtime, not compile time.

The negatives of not using an ORM is far worse than the negatives of not reigning in some developers who shouldn't be making complex queries.

If they don't even know how to check the SQL their complex ORM query produces, that's a training problem, not an ORM problem.

It's one of our great weaknesses as a profession, assuming everyone will figure stuff out on their own.

Re: Good system design

#363

Post by frankc »

avinassh wrote: post #361

OP has this first: > If a system has distributed-consensus mechanisms, many different forms of event-driven communication, CQRS, and other clever tricks, I wonder if there’s some fundamental bad decision that’s being compensated for (or if the system is just straightforwardly over-designed). then later down in the article: > Send as many read queries as you can to database replicas. A typical database setup will have…

Somewhat, though CQRS might advocate for separate read and write models. It might be something like the writer publishes an event that is consumed by something that updates the read model and queries go directly to the read model. Whether or not that is overengineering depends on the problem and load.

Re: Good system design

#364

Post by cryptonector »

belZaah wrote: post #347
cryptonector wrote: post #329

Earlier quoted context omitted.

Really! I've written about this in my comments here... Here's a brief summary: - typical thread-per-client programming is terribly wasteful because it needs large stacks that must be able to grow (within reason), and this leads programmers to smear client state all over the stack, which then means that the memory and _cache_ footprint of per-client state is huge even though the state is highly compressible, and this…

There’s also a systems level rationale to this. Without good isolation, you’ll get a feedback loop: threads start to step on each other’s toes. This leads to slower response times. Which, at a given request pressure, leads to more parallel threads. Which slows them down even more. If there’s a brief peak in pressure, that drops the response time below a critical point, such a system will never recover and you’ll get…

Yes, and thus circuit breakers. By sizing offered capacity to some factor of actual capacity you can limit the effects of too much demand to causing backpressure naturally (rejecting requests) instead of timeouts and retries. This then allows you some level of access -- such as to your health and diagnostics end-points, because CPU usage doesn't become so high that you can't even run those.

Re: Good system design

#365

Post by cryptonector »

jackblemming wrote: post #317
mkozlows wrote: post #254

Earlier quoted context omitted.

(For context, I've conducted hundreds of system design interviews and trained a dozen other people on how to do them at my company. Other interviewers may do things differently or care about other things, but I think what I'm saying here isn't too far off normal.) I think three things about what you're saying: 1. The answers you're giving don't provide a lot of signal (the queue one being the exception). The question…

All that “signal” nonsense can be parroted by both an LLM and someone who read “how to pass system interviews”. Yea, great “signal”.

Not really, not in live, oral interviews.

Though I once had a case of the person we thought we were hiring and the person we got being different people. The fix for that is to always have one final in-person interview.

Re: Good system design

#366

Post by quails8mydog »

avinassh wrote: post #361

OP has this first: > If a system has distributed-consensus mechanisms, many different forms of event-driven communication, CQRS, and other clever tricks, I wonder if there’s some fundamental bad decision that’s being compensated for (or if the system is just straightforwardly over-designed). then later down in the article: > Send as many read queries as you can to database replicas. A typical database setup will have…

I thought cqrs was a code pattern where you segregate query models from command models, rather than something that specifies where the data is read from or is written to.

Re: Good system design

#367

Post by Juliate »

mkozlows wrote: post #254
alixanderwang wrote: post #131

> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…

(For context, I've conducted hundreds of system design interviews and trained a dozen other people on how to do them at my company. Other interviewers may do things differently or care about other things, but I think what I'm saying here isn't too far off normal.) I think three things about what you're saying: 1. The answers you're giving don't provide a lot of signal (the queue one being the exception). The question…

What you're describing is how the interview process _is_ disconnected from the actual needs, and how it's good to literally "play the game" to get in.

But on the other side, that kind of interview process is itself also a signal candidates might take to avoid playing the game, knowing that most (not all, of course) companies probing for the wrong signals during the interview process are indicative of how they do function as a whole.

(been in both types of companies, and in both sides of the table)

Re: Good system design

#368

Post by maxbond »

mrkeen wrote: post #346
RaftPeople wrote: post #231

Earlier quoted context omitted.

> Audit tables are a dumb concept because they imply bolting on an actual source of truth in addition to the regular not so source of truth tables, The regular table is the source of truth, the audit table is just a historical record of what changed and when.

So what do you do if a balance has $30 and the audit table shows two deposits of $20?

The same as if you didn't have an audit table and the balance was wrong. You declare some kind of incident and investigate until you find this critical bug. Hopefully you can use the audit trail to determine what the right answer is and fix everyone's balance.

How do you know which one to trust? By reading the code and figuring out what the bug is.

Re: Good system design

#369

Post by maxbond »

setr wrote: post #308
maxbond wrote: post #288

Earlier quoted context omitted.

Setting a boolean is, quite often, an important event you want to keep track of. (Specifically, when it is a flag indicating an event took place, eg, once it's been set it is rarely if ever unset.) Of course it isn't universally applicable to every boolean column; nothing is. Of course you need to understand your schema and the problem you're solving. That doesn't make it mindless or make it any less valuable. It is…

The original premise, from TFA, is Another is the Twitter-optimized “you’re a terrible engineer if you ever store booleans in a database” clever trick That is, universal application. My point is: doing that, for the most part, will not the track the things you actually want, because bools are not the driving decision maker on value of tracking (and many things that need to be tracked will not be bools). It may accide…

I think we agree more than we disagree on how to go about designing a schema, but that my narrow disagreement is that I think a.) timestamps and enums are better choices than booleans most of the time (without judging anyone who uses booleans to be a "terrible engineer", it's not malpractice) and b.) that's a legitimate trick of the trade rather than something "mindless".

I would agree that throwing a table together based on Twitter vibes without actually designing it will only work like a stuck clock, but I think that goes without saying. I kinda disregard opinions attributed to nebulous Twitter posters and not specific people. I think when you conflate people's opinions together like that you reduce their position to a caricature. If you poll that group of people almost all of them will have a more nuanced opinion.

Re: Good system design

#370

Post by RaftPeople »

mdavid626 wrote: post #248
RaftPeople wrote: post #221

Earlier quoted context omitted.

> In any case, I believe a DB per backend service isn't a decision driven by the frontend - rather, it's driven by data migration and data access requirements. I think the idea of breaking up a shared enterprise DB into many distinct but communicating and dependent DB's was driven by a desire to reduce team+system dependencies to increase ability to change. While the pro is valid and we make use of the idea sometimes…

I disagree. I generally understand the problem a "split-up" database brings to the table. This is how people designed things in the last many decades. What I propose is to leave this design behind. The split up design fits modern use cases much better. People want all kind of data. They want to change what data they want rather often. "One" database for all of this doesn't really work -- you can't change the schema s…

> In the split-up design, since you're not sharing the database, you can do whatever you want.

> we can't add a new field, we can't change this field

Ok, let's do an example.

Assumption:

A-ERP system with approximately 30 modules in use (e.g. sales order mgmt, inventory, purchasing, etc)

B-For split DB, the DB is split by module and data flows exist for all shared data. So there are X different copies of the item master (many and possibly most of those modules use the item master), each with the subset of data required by the specific module.

Sample change, add a new field to the item master:

Shared DB:

1-Update DB schema for item master

2-Update code in different modules that need to use the new data element (per feature requirements)

Split DB:

1-Update DB schema in all modules that require the new data element (per feature requirements)

2-Update code in different modules that need to use the new data element

3-Update the data flows for item data in each module that needs to use the new data element

I think you're understating the level of effort when you say "now we can do whatever we want". The actual effort in this change (which is a very common example) is actually greater than in a shared DB and requires more coordination.

Again, there are times when it's the right thing to do, but definitely not a silver bullet without trade-offs.

Post reply on HN