Good system design • Hacker News Board

Re: Good system design

#151

Post by paffdragon » Sat, Aug 16, 2025, 2:05 PM UTC

wahnfrieden wrote: post #140

sethammons wrote: post #123

Earlier quoted context omitted.

The goal is to minimize what needs changing when things need changing. When you need to alter the datastore, usually for product or scalability, you have to orchestrate all access to that datastore. Ergo: one only one thing using the datastore means less orchestration. At work, we just updated a datastore. We had to move some tables to their own db. 3 years later, 40+ teams have updated their access. This was a produ…

A reused code library for DB use is an alternative there

That moves your API layer to the client library you need to distribute and build for your customers in programming languages they support. There are some cases where a thick client makes sense, but usually easier to do it server side and let customers consume the API from their env, it is easier to patch the server than to ship library updates to all users.

Re: Good system design

#152

Post by ramraj07 » Sat, Aug 16, 2025, 2:05 PM UTC

alixanderwang wrote: post #131

> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…

Do you _want_ to work in these places? In my experience, if they expect you to run kube using kube in the interview, thats exactly what they do in their ststems as well.

Re: Good system design

#153

Post by gethly » Sat, Aug 16, 2025, 2:09 PM UTC

mexicocitinluez wrote: post #115

gethly wrote: post #66

Actually event-sourcing solves most of the pains - events, schema, push/pull, caching, distribution... whatever. The downside is that it is definitely not suitable for small projects and the overhead is substantial(especially during the development stage when you want to ship the product as soon as possible). On the other hand, once you get it going, it's an unstoppable beast.

There are some tools that try to solve this (MartenDb, for instance), but I wish there was an easier to way to integrate a system where some parts us ES and some parts don't. Almost all the tools I've seen are either fully event-sourced or have nothing to do with event-sourcing. There aren't a ton of in-betweens.

ES is at the core of the system where it is being used, so there really is no in-between option here. I've built two production ES systems and I was toying with few ideas to make it more DX friendly by using json as core of any entity/object and using json patch for events but in the end it made no sense because ES must be absolutely strict on schema, which evolves over time, and data types. You must be able to process events that might be a decade old and for objects that no longer exist. There is no wiggle room. Hence the aforementioned overhead. I do not know MartenDb but in essence every db today uses ES as that is how transactions work, except the event log is discarded after the commit. But either way, ES on db level is meaningless, except maybe being able to use it as actual log for auditing purposes but you won't be able to process it by any means as schema changes over time and db schema has little to do with the application itself anyway.

Re: Good system design

#154

Post by Yokohiii » Sat, Aug 16, 2025, 2:13 PM UTC

loglog wrote: post #135

Yokohiii wrote: post #119

Earlier quoted context omitted.

I've only once tried to use stored procedures in mysql and it was almost impossible to debug back then. Very painful. Average devs already have issues being smart with their databases and stored procedures would add to that. Stored procedures also add another risk. You have to keep them in sync with code, making releases more error prone. So you have to add extra layers of complexity to manage versioning. I can see t…

> Stored procedures also add another risk. You have to keep them in sync with code, making releases more error prone. This one is easily solved: never change a stored procedure. Every version should get a new name.

That's what I meant when I've mentioned versioning.

Re: Good system design

#155

Post by feyman_r » Sat, Aug 16, 2025, 2:13 PM UTC

If you want to learn more about good system design at an abstract level (not just online), cannot recommend Systemantics[1] by John Gall enough. I wish all engineers get an opportunity to read it.

[1] https://en.m.wikipedia.org/wiki/Systemantics

Re: Good system design

#156

Post by Yokohiii » Sat, Aug 16, 2025, 2:19 PM UTC

tossandthrow wrote: post #134

Yokohiii wrote: post #121

Earlier quoted context omitted.

In my ears that's just neglect? You assume your ORM does the basic data mapping right and don't verify it?

No? The difference is to verify it ones for the orm VS ones for every single place your query.

A raw query doesn't has to be repeated in every place it's required. Not sure what your point is.

Re: Good system design

#157

Post by dondraper36 » Sat, Aug 16, 2025, 2:22 PM UTC

feyman_r wrote: post #155

If you want to learn more about good system design at an abstract level (not just online), cannot recommend Systemantics[1] by John Gall enough. I wish all engineers get an opportunity to read it. [1] https://en.m.wikipedia.org/wiki/Systemantics

I enjoyed reading this book (it's a short one), even though the prose is very, well, special :)

Re: Good system design

#158

Post by sgarland » Sat, Aug 16, 2025, 2:29 PM UTC

motorest wrote: post #92

What a great article. It's always a treat to read this sort of take. I have some remarks though. Taken from the article: > Avoid having five different services all write to the same table. Instead, have four of them send API requests (or emit events) to the first service, and keep the writing logic in that one service. This is not so cut-and-dry. The trade offs are far from obvious or acceptable. If the five services…

> Additionally, having five services accessing the same database is a code smell.

Counterpoint (assuming by database you mean database cluster, not a schema): having a separate physical DB for each service means that for most places, your reliability has now gone from N to N^M.

Re: Good system design

#159

Post by maxbond » Sat, Aug 16, 2025, 2:30 PM UTC

oftenwrong wrote: post #90

ZYbCRq22HbJ2y7 wrote: post #13

> You’re supposed to store timestamps instead, and treat the presence of a timestamp as true. I do this sometimes but not always - in my view there’s some value in keeping a database schema immediately-readable. Seems overly negative of broad advice on a good pattern? is_on => true on_at => 1023030 Sure, that makes sense. is_a_bear => true a_bear_at => 12312231231 Not so much, as most bears do not become bears at som…

A boolean is smaller, which is a relevant consideration for some workloads. For example, you may be pre-aggregating a large amount of data to serve a set of analytical queries which do not care about the associated timestamp. The smaller data type is more efficient both in storage and in query execution. Additionally, there are situations where it is logical to store a boolean. For example, if the boolean denotes an…

It's unlikely the boolean will result in better utilization, the savings will probably be consumed by padding. Most people don't know how to use structure packing to create a row which is actually smaller after it's been padded (though it's not very hard, anyone could learn). Columns are generally ordered by which features were shipped first and not by alignment (as is necessary to minimize padding).

I do try my best to pack my columns, but it's a fragile and likely premature optimization. Better to opt for something defensive at a cost of like, 7 bytes per row (in Postgres).

Re: Good system design

#160

Post by zellyn » Sat, Aug 16, 2025, 2:34 PM UTC

alixanderwang wrote: post #131

> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…

I recently had an interview like this. Felt like half the answers I gave were of the form, “You can do scaling/sharding/partitioning thing X here, but once again, for an internal app I’d try really hard to avoid doing any of that”. If you’re interviewing with capable, experienced developers, they’ll appreciate that answer (at least, I got the offer on this one!)