Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good system design

seangoedecke.com

Post reply on HN
Print view View original

121–130 of 400 posts

Re: Good system design

#121

Post by Yokohiii »

tossandthrow wrote: post #113
Yokohiii wrote: post #106

Earlier quoted context omitted.

I don't understand why all these problems should be easier handled with an ORM then with raw sql?

It is a granluarity tradeoff. With SQL you need to explicitly test all queries where the shape granularity is down to field level. When you map data onto an object model (in the dto sense, not oop sense) you have bigger building blocks. This gives a simpler application that is more reliable. Obviously you need to pick a performant orm - and it seems a lot of people in these threads have been traumatized. Personally,…

In my ears that's just neglect? You assume your ORM does the basic data mapping right and don't verify it?

Re: Good system design

#122

Post by victorbjorklund »

bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

Not sure I agree. First of all it can be more performant. Say you fetch 1000 records. And we need to join on a table where these 1000 records just got 2 different foreign keys. Instead of joing in db and fetching a lot more data we can do two queries and join in app instead. Secondly, makes it easier to cache data. Lets say the thing we joing with almost never changes (like some country info) we can cache that and just join it with the data from the db.

Not saying this should always be the case, but sometimes it is the right call.

Re: Good system design

#123

Post by sethammons »

motorest wrote: post #92

What a great article. It's always a treat to read this sort of take. I have some remarks though. Taken from the article: > Avoid having five different services all write to the same table. Instead, have four of them send API requests (or emit events) to the first service, and keep the writing logic in that one service. This is not so cut-and-dry. The trade offs are far from obvious or acceptable. If the five services…

The goal is to minimize what needs changing when things need changing.

When you need to alter the datastore, usually for product or scalability, you have to orchestrate all access to that datastore.

Ergo: one only one thing using the datastore means less orchestration.

At work, we just updated a datastore. We had to move some tables to their own db. 3 years later, 40+ teams have updated their access. This was a product need. If this was a scale issue, the product would just have died sans some as of yet imagined solution.

Re: Good system design

#124

Post by teraflop »

victorbjorklund wrote: post #122
bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

Not sure I agree. First of all it can be more performant. Say you fetch 1000 records. And we need to join on a table where these 1000 records just got 2 different foreign keys. Instead of joing in db and fetching a lot more data we can do two queries and join in app instead. Secondly, makes it easier to cache data. Lets say the thing we joing with almost never changes (like some country info) we can cache that and ju…

But as a counterpoint to that, (a) the database has its own caching built in, which you don't have to implement, and (b) the database knows when to invalidate its cache.

To quote Douglas Adams: "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair."

Likewise, if you cache a piece of data in your application because you assume that it won't change, that just makes it likely that if and when it does change, you'll have bugs. Moving the cache to the database layer so that it can be properly invalidated fixes this.

It's true that an application-side join can still be more performant if the DB cache isn't good enough, but IMO you should only take that step after actually profiling your queries.

Re: Good system design

#125

Post by codr7 »

Replacing booleans with timestamps might be a good idea sometimes, presenting it as The Solution isn't very constructive imo.

Adding a separate table where the presence of a record means 'true' allows recording related state without complicating the main table.

And sometimes a boolean is exactly what you want.

Re: Good system design

#126

Post by msiyer »

> Avoid having five different services all write to the same table. Instead, have four of them send API requests (or emit events) to the first service, and keep the writing logic in that one service.

The ideal solution: Avoid having five different services all write to the same table.

If five different services have to write to the same table, there is a major overlap of logic too. Are the five services really different or one would suffice?

Taking practical realities into consideration, we can do what the author says. However, we risk implementing a lot of orchestration logic. We introduce a whole new layer of problems. Is that time not better spent refactoring the services: either give them their own DB tables or merge them into one servic?

Re: Good system design

#127

Post by wavemode »

codr7 wrote: post #125

Replacing booleans with timestamps might be a good idea sometimes, presenting it as The Solution isn't very constructive imo. Adding a separate table where the presence of a record means 'true' allows recording related state without complicating the main table. And sometimes a boolean is exactly what you want.

the article presents that as an example of bad advice

Re: Good system design

#128

Post by Muromec »

motorest wrote: post #92

What a great article. It's always a treat to read this sort of take. I have some remarks though. Taken from the article: > Avoid having five different services all write to the same table. Instead, have four of them send API requests (or emit events) to the first service, and keep the writing logic in that one service. This is not so cut-and-dry. The trade offs are far from obvious or acceptable. If the five services…

>And what exactly do you buy yourself? More failure modes and a higher micro services tax?

Nice boxes in the architectural diagram. Each box is handed to a different team and then, when engineers from those teams don't talk to each other, the system doesn't suddenly fail in an unexpected way.

Re: Good system design

#129

Post by graphviz »

whodidntante wrote: post #91

Never write an article about good system design. In all seriousness, this is an extraordinary subtle and complex area, and there are few rules. For example, "if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory" may be useful in certain circumstances. For highly scalable consumer systems, the rule of "avoid joins as much as possible" can work a lot…

And by "system" we mainly meant "transactional website."

Re: Good system design

#130

Post by esafak »

StevenWaterman wrote: post #58

What do you call system design, when it's referring to the design of systems in general, and not just computer services? As in: - writing a constitution - designing API for good DX - improving corporate culture I intuitively want to call all of those system design, because they're all systems in the literal sense. But it seems like everyone else uses "system design" to mean distributed computer service design. Any id…

Organization design. https://en.wikipedia.org/wiki/Organizational_architecture
Post reply on HN