Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good system design

seangoedecke.com

Post reply on HN
Print view View original

381–390 of 400 posts

Re: Good system design

#381

Post by jiggawatts »

belZaah wrote: post #347
cryptonector wrote: post #329

Earlier quoted context omitted.

Really! I've written about this in my comments here... Here's a brief summary: - typical thread-per-client programming is terribly wasteful because it needs large stacks that must be able to grow (within reason), and this leads programmers to smear client state all over the stack, which then means that the memory and _cache_ footprint of per-client state is huge even though the state is highly compressible, and this…

There’s also a systems level rationale to this. Without good isolation, you’ll get a feedback loop: threads start to step on each other’s toes. This leads to slower response times. Which, at a given request pressure, leads to more parallel threads. Which slows them down even more. If there’s a brief peak in pressure, that drops the response time below a critical point, such a system will never recover and you’ll get…

Most systems cap the size of their thread pool and put excess requests into a queue.

Re: Good system design

#382

Post by jiggawatts »

ndriscoll wrote: post #302
yojo wrote: post #246

Earlier quoted context omitted.

If a candidate doesn’t ask clarifying questions that lead them to an understanding of QPS, storage requirements, and throughput considerations, that’s a mark against. At that point, if you want to see them design a distributed system with all the bells and whistles, you should stop them, tell them the kind of traffic they need to handle, then let them go again. If they persist in designing a system that cannot handle…

The problem with this is people seem to have mismatched understandings of what a single system can handle. e.g. my 8 year old quad core i5 desktop with a bit of batching optimization can handle 5 digit requests per second with 15 ms p99 with some nontrivial application logic doing several joins. I don't think I've tried that same benchmark on a modern minipc, but I expect it should be similar. That's well above what…

“Prove that you can apply solutions to yesterday’s problems today.” is a good strategy except in industries where today is exponentially different to yesterday.

Re: Good system design

#383

Post by 5Qn8mNbc2FNCiVV »

sgarland wrote: post #332
maest wrote: post #257

> But in most cases replication lag can be worked around with simple tricks: for instance, when you update a record but need to use it right after, you can fill in the updated details in-memory instead of immediately re-reading after a write. I get it, but that sounds very finicky code to get right and a good source of hard-to-debug bugs.

Two things: 1. Essentially every RDBMS except MySQL has a RETURNING clause, so you can read your write essentially for free (and even MySQL lets you read the auto-increment value it inserted). 2. Barring that, how is this finicky to get right? If you get an ack back from the DB, assuming you haven’t done silly things to fsync settings, it’s written. It’s durable. So, within the same try/except or similar, take what y…

You'd have to reach the same instance for your next request. Easier to just tell clients that they get returned a header that they should attach to their requests so the backend can route their reads to the primary for some time until replication caught up.

Re: Good system design

#384

Post by __turbobrew__ »

try_the_bass wrote: post #309
__turbobrew__ wrote: post #260

Earlier quoted context omitted.

If you want to get top dollar at a FAANG you will need to go through these type of system design interviews. You could say you shouldn’t work for a FAANG which is fair, but FAANG pays top dollar.

So if you're after the FAANG money, you have to play the FAANG interview game. If you're unwilling to play the FAANG interview game, then maybe you shouldn't be pursuing FAANG money.

Doing FAANG interviews is a necessary, but not sufficient condition to getting a FAANG salary. As others have said, non-FAANGs are trying to do FAANG interviews, and I would tell them to pound sand.

Re: Good system design

#385

Post by ZYbCRq22HbJ2y7 »

grey-area wrote: post #348
ZYbCRq22HbJ2y7 wrote: post #315

Earlier quoted context omitted.

Yeah, need a bit of imagination to walk there with me, but you are saying just use `saw_eclipse_at`. That would require having knowledge of the entity's birthday, the date that the event occurred on, and so on, which in this imaginary scenario, we do not.

I would not store that in a schema, storing bday and date seen is much more useful so that when the business inevitably asks for saw eclipse at 50 too you can answer the question without adding a saw at 50 boolean. Bday is also super useful info for a business that cares how old someone is (as your hypothetical one clearly does). Often the stated requirements of a problem are far too specific. Part of the job is syst…

> This example is a great illustration of why booleans are usually a mistake.

It is not. You are ignoring hypothetical constraints to bolster your own hypothetical argument, which is apparently some permutation of, "tell people they are wrong, and do some imputing of imaginary data. repeat until you have ideal 'system design'"

Re: Good system design

#386

Post by ZYbCRq22HbJ2y7 »

moebrowne wrote: post #29
setr wrote: post #17

Earlier quoted context omitted.

If you take the statement at face value — essentially storing booleans in the db ever is a bad smell - then he’s correct. Although I’m not even sure it’s broadly a good principle, even in the on_at case; if you actually care about this kind of thing, you should be storing it properly in some kind of audit table. Switching bool to timestamp is more of a weird lazy hack that probably won’t be all that useful in practic…

It's well documented that soft delete is more of a headache than it's worth https://brandur.org/soft-deletion

Maybe well opined on, not well-documented.

There are clients (ORM, etc) that handle soft deletion just fine in many of the scenarios laid out in that article.

Re: Good system design

#387

Post by tossandthrow »

radlad wrote: post #281
tossandthrow wrote: post #133

Earlier quoted context omitted.

While this api in particular is not publicly exposed, that would not be a concern. The key is to hold the same schema on the database as on the graphql and use tooling that can translate a gql query into a single query.

The issue I've seen with GraphQL isn't necessarily the count of queries run, but rather the performance or said queries (i.e. most SQL queries are not performant without proper indexes for the specific use case, but GraphQL allows lots of flexibility in what queries users can run.)

Yes - one needs to ensure that the data is well indexed - that is reasonable.

But indices does not need to yield a single result. It is OK that indices reduce the result set to tens or couple of hundreds of result. That is well within the performance requirements (... of our app)

Re: Good system design

#388

Post by DanielHB »

bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

Microservice achitecture promotes splitting data cross multiple databases making it impossible to do proper DB JOINs from application code.

Then companies buy a solution to aggregate all the different databases in a single "data-lake" (or whatever buzzword is hot right now) so you can do OLAP queries. Without consistency guarantees of course.

And I am not saying this is never the _right_ solution, but it should almost never be the _first_ solution

Re: Good system design

#389

Post by DanielHB »

dondraper36 wrote: post #57
mattmanser wrote: post #51

Earlier quoted context omitted.

Most ORMs will happily let you map stored procedures and views to a class, you can have as many models as you want. So your point doesn't really make sense. The author's said nothing about ORMs. It feels like you're trying to post a personal beef about ORMs that's entirely against the "pragmatic" software design engineering the author's opining. Using ORMs to massively reduce your boiler-plate CRUD code, then using r…

In Go, for example, there is a mixed approach of pgx + sqlc, which is basically a combo of the best Postgres driver + type-safe code generator (based on raw SQL). https://brandur.org/sqlc Even though I often use pgx only, for a new project, I would use the approach above.

I did some exploratory analysis on sqlc some time ago and I couldn't for the life of me figure out how to parametrize which column to sort-by and group-by in queries.

It is quite neat, but I don't think it actually replaces a proper ORM for when ORMs are actually useful for. That on top of all the codegen pitfalls.

I personally quite like the Prisma approach which doesn't map database data to objects, but rather just returns an array of tuples based on your query (and no lazy loading of anything ever). With typescript types being dynamically computed based on the queries. It has its own pitfalls as well (like no types when using raw queries).

Re: Good system design

#390

Post by dennis_jeeves2 »

cryptonector wrote: post #321
philjohn wrote: post #286

Earlier quoted context omitted.

+1 on the signal. A great candidate won't need you to pry further than asking about backpressure, they'll explain WHY it's not necessary for the qps, what qps it would start becoming necessary, and how they would build it into their design down the line if the service takes off. One of the things I tell people preparing for system design interviews is the more senior you are, the more you need to drive the interview…

As the interviewee I'd use such a question to demonstrate my knowledge of the topic. For this question I'd point out that thread-per-CPU w/ async I/O designs w/ maximum live connections and clever connection pool acceptance & eviction policies, and limited buffering, together intrinsically limit oversubscription, but I would still talk extensively about health monitoring and external circuit breaking, as well as the…

Lol
Post reply on HN