Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good system design

seangoedecke.com

Post reply on HN
Print view View original

341–350 of 400 posts

Re: Good system design

#341

Post by paulddraper »

Swizec wrote: post #192
Aurornis wrote: post #178

Earlier quoted context omitted.

> I want you to start with these answers then we can layer on complexity if you’ve solved the problem and there’s time left to go into navel gazing mode Exactly. Part of the interview is explaining when and why these techniques are necessary as part of demonstrating your understanding. If the candidate gives non-answers like “I don’t think it matters because you’re a startup” or “I’d just use whatever database I’m co…

> The interview is your chance to show you know the topics and when to apply them, not the time to argue that the startup shouldn’t care about such matters. A good way to answer these, I think, is some version of ”We probably won’t run into these issues at the scale we’re talking about, but when we run into A, B, C problems, we can try X, Y, Z solutions.” This shows that you’re making a conscious tradeoff and know wh…

Right, and you might be small in $year but presumably you expect to grow and they don’t want to replace the team because they can’t think how to operate in any other circumstances.

Re: Good system design

#342

Post by mrkeen »

tremon wrote: post #190
mrkeen wrote: post #55

Earlier quoted context omitted.

Audit tables are a dumb concept because they imply bolting on an actual source of truth in addition to the regular not so source of truth tables , and only if the programmer gets around to it (like documentation or logging or whatever else falls along the wayside).

This doesn't make sense to me-- if the regular tables don't capture the true state, then the audit tables based on them will not magically become a source of truth either.

Exactly. If they agree, then there was no point to introducing a second source of truth. If they disagree, then which one do you trust?

Re: Good system design

#343

Post by belZaah »

When systems people talk about system design, they talk about the whole value-generating system and not just software or hardware. This typically involves people and brings in issues like the Conway’s Law. A tightly coupled team with little independence in terms of workstreams will produce a monolith regardless of what the architect dreams of, for example. If you have two sets of users with diverging regulatory and organizational needs (people who maintain the registry of citizens vs people who issue identity documents), you will have two separate data stores regardless of how much sense it makes to have just one.

Re: Good system design

#344

Post by throwawaythekey »

nostrademons wrote: post #218
_fat_santa wrote: post #169

Earlier quoted context omitted.

This goes back to "interviews go both ways". All those answers you gave are very reasonable and if I was your interviewer I'd pass you with flying colors. On the other hand if you're interviewing at a place that doesn't pass you with flying colors for those responses, that really says more about them than it does about you and may not be a great place to work. But to your point, many times one interviews for a job th…

If I were the interviewer, I'd try to adjust the problem statement with some hypotheticals to tease out their depth of knowledge: > "That's not really worth considering for this amount of QPS" "What if Michael Jackson dies and your (search|news|celebrity gossip) service gets a spike in traffic way beyond the design parameters? How would you anticipate and mitigate such an event?" (Extra points if the answer is not ne…

>I would've expected the candidate to at least be able to talk about indexing, tradeoffs of joining in the DB vs. in the application, schema migrations and upgrades, creating separation between data-at-rest vs. data-in-flight, etc.

The problem is that many of these trade-offs only applied to older databases. The more relevant axis is about how distributed the db is, the replication type etc.

Re: Good system design

#345

Post by throwawaythekey »

Aurornis wrote: post #178
Swizec wrote: post #162

Earlier quoted context omitted.

> These are not the answers they're looking for. These ARE the answers we are looking for. As the system design interview (I’ve done hundreds) I want you to start with these answers then we can layer on complexity if you’ve solved the problem and there’s time left to go into navel gazing mode. Seeing the panic slowly build in mid-level engineers’ eyes as it dawns on them that not every problem can be solved by cachin…

> I want you to start with these answers then we can layer on complexity if you’ve solved the problem and there’s time left to go into navel gazing mode Exactly. Part of the interview is explaining when and why these techniques are necessary as part of demonstrating your understanding. If the candidate gives non-answers like “I don’t think it matters because you’re a startup” or “I’d just use whatever database I’m co…

> Part of the interview is explaining when and why these techniques are necessary as part of demonstrating your understanding.

The slightly altered "explain when and why these techniques are *not* necessary" is much less appreciated.

Re: Good system design

#346

Post by mrkeen »

RaftPeople wrote: post #231
mrkeen wrote: post #55

Earlier quoted context omitted.

Audit tables are a dumb concept because they imply bolting on an actual source of truth in addition to the regular not so source of truth tables , and only if the programmer gets around to it (like documentation or logging or whatever else falls along the wayside).

> Audit tables are a dumb concept because they imply bolting on an actual source of truth in addition to the regular not so source of truth tables, The regular table is the source of truth, the audit table is just a historical record of what changed and when.

So what do you do if a balance has $30 and the audit table shows two deposits of $20?

Re: Good system design

#347

Post by belZaah »

cryptonector wrote: post #329
iamcreasy wrote: post #325

Earlier quoted context omitted.

I beg you to write an article expanding on this points. I'll pay to read this article.

Really! I've written about this in my comments here... Here's a brief summary: - typical thread-per-client programming is terribly wasteful because it needs large stacks that must be able to grow (within reason), and this leads programmers to smear client state all over the stack, which then means that the memory and _cache_ footprint of per-client state is huge even though the state is highly compressible, and this…

There’s also a systems level rationale to this. Without good isolation, you’ll get a feedback loop: threads start to step on each other’s toes. This leads to slower response times. Which, at a given request pressure, leads to more parallel threads. Which slows them down even more. If there’s a brief peak in pressure, that drops the response time below a critical point, such a system will never recover and you’ll get a server furiously computing without an apparent reason only to behave normally after a restart.

Re: Good system design

#348

Post by grey-area »

ZYbCRq22HbJ2y7 wrote: post #315
grey-area wrote: post #263

Earlier quoted context omitted.

I’ve never seen anything remotely like that in a schema and it seems inappropriate anyway - those should IMO be timestamps like saw_eclipse_at not booleans. You should not encode business rules in the schema (like certain magic dates) because those business rules always change over time.

Yeah, need a bit of imagination to walk there with me, but you are saying just use `saw_eclipse_at`. That would require having knowledge of the entity's birthday, the date that the event occurred on, and so on, which in this imaginary scenario, we do not.

I would not store that in a schema, storing bday and date seen is much more useful so that when the business inevitably asks for saw eclipse at 50 too you can answer the question without adding a saw at 50 boolean. Bday is also super useful info for a business that cares how old someone is (as your hypothetical one clearly does).

Often the stated requirements of a problem are far too specific. Part of the job is system design is saying no to suggestions and finding the deeper constraints.

This example is a great illustration of why booleans are usually a mistake.

Re: Good system design

#349

Post by spectraldrift »

In general, I agree with the author's presumption that simplicity is better in system design. The frequent and tired allergy for managed queues, however, does not follow.

> Sometimes you want to roll your own queue system.

I have never wanted to do this.

> For instance, if you want to enqueue a job to run in a month, you probably shouldn’t put an item on the Redis queue.

This specific requirement sounds more like a cron job use case, not a queue case.

> In this case, I typically create a database table for the pending operation with columns for each param plus a scheduled_at column. I then use a daily job to check for these items with scheduled_at At this point I've decided the author doesn't understand when or why to use a queue. For a strictly scheduled event which must occur on day = (today + N), the proposed approach seems fine. However if you use this for a typical queue use-case you will end up reimplementing a queue in your database (poorly). This is typically far more complex than just using a managed queue service if it's available to you. By more complex, I mean more lines of code and more oncall burden. Furthermore, if you grow, a managed queue is often something that needs very little hand holding. Queues are great because they are simple- both at low and high volume. They are a technology that "just works", and often provide a lot of helpful features out of the box.

I don't know the author, but I've encountered this engineering philosophy before. It's a perspective often held by engineers whose ideas haven't been stress-tested by the long-term realities of a successful business.

It's the kind of advice you can sell to the 99% of startups that fail, and because they fail for other reasons, you never have to be proven wrong.

Re: Good system design

#350

Post by manoji »

Excellent article . Few more that come to mind

Think real carefully about breaking transactionality unless absolutely needed . It has been the single source of most problems i have seen over the past few years.

Keeping 2 different systems in sync is really hard do not do it if you dont have a real need to do it.

Monoliths are really good , there is absolutely no need to run microservices or any services for that matter other than a single monolith. The place where i work at is generating billions of dollars with a single monolith.Having said that there will come a time when some logic has to go to a different services , If you get there your company is really really successful :) .

Relational databases can do a lot more than what you think and they can absolutely scale well.

Post reply on HN