Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Good system design

seangoedecke.com

Post reply on HN
Print view View original

111–120 of 400 posts

Re: Good system design

#111

Post by anal_reactor »

nvarsj wrote: post #80

> Paradoxically, good design is self-effacing: bad design is often more impressive than good. Rings very true. Engineers are rated based on the "complexity" of the work they do. This system seems to encourage over-engineered solutions to all problems. I don't think there is enough appreciation for KISS - which I first learned about as an undergrad 20 years ago.

This is unfortunately true. People love complex solutions, and suggesting a simple one usually comes across as incompetent, while the reality is, simple solutions are easy to manage, which ensures the success of the project as a whole.

Sure, there are problems that are inherently complex and require complex solutions. But most likely yours isn't one of them, most likely you have a basic web app.

Re: Good system design

#112

Post by mexicocitinluez »

bob1029 wrote: post #24
bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

This is a big part of what makes ORMs a problem. Writing raw SQL views/queries per MVC view in SSR arrangements is one of the most elegant and performant ways to build complex web products. Let the RDBMS do the heavy lifting with the data. There are optimizations in play you can't even recall (because there's so many) if you're using something old and enterprisey like MSSQL or Oracle. The web server should be able to…

>The typical ORM implementation is the exact opposite of this - one strict object model that must be used everywhere. It's about as inflexible as you can get.

I can't respond to the "typical" part as most of my experience is using EF Core, but it's far from inflexible.

Most of my read-heavy, search queries are views I've hand written that integrate with EF core. This allows me to get the benefit of raw SQL, but also be able to use LINQ to do sorting/paging/filtering.

Re: Good system design

#113

Post by tossandthrow »

Yokohiii wrote: post #106
tossandthrow wrote: post #38

Earlier quoted context omitted.

Have you ever build a complex app like this? In particular, have you have to do testing, security (eg. row level security), manage migrations, change management (eg. for SOC2 or other security frameworks), cache offloads (Redis, and friends), support for microservices, etc. Comments like this give me a vibe of young developers trying out Supabase for the first time feeling like that approach can scale indefinitely.

I don't understand why all these problems should be easier handled with an ORM then with raw sql?

It is a granluarity tradeoff.

With SQL you need to explicitly test all queries where the shape granularity is down to field level.

When you map data onto an object model (in the dto sense, not oop sense) you have bigger building blocks.

This gives a simpler application that is more reliable.

Obviously you need to pick a performant orm - and it seems a lot of people in these threads have been traumatized.

Personally, I run a complex application where developers freely use a graphql schema and requests are below 50ms p99 - gql in translated into joins by the orm, so we do not have any n+1 issues, etc.

Re: Good system design

#114

Post by Yokohiii »

mattmanser wrote: post #51
bob1029 wrote: post #24

Earlier quoted context omitted.

This is a big part of what makes ORMs a problem. Writing raw SQL views/queries per MVC view in SSR arrangements is one of the most elegant and performant ways to build complex web products. Let the RDBMS do the heavy lifting with the data. There are optimizations in play you can't even recall (because there's so many) if you're using something old and enterprisey like MSSQL or Oracle. The web server should be able to…

Most ORMs will happily let you map stored procedures and views to a class, you can have as many models as you want. So your point doesn't really make sense. The author's said nothing about ORMs. It feels like you're trying to post a personal beef about ORMs that's entirely against the "pragmatic" software design engineering the author's opining. Using ORMs to massively reduce your boiler-plate CRUD code, then using r…

The way you describe it, it would be ideal if ORMs would only handle very basic CRUD and force you to use raw sql for complex queries. But that's not reality and not how they are used, not always. In my opinion some devs take pride to do everything with their favorite ORM.

I think if an app uses 90% ORM code with the remains as raw queries, a junior is inclined to favor ORM code and is also less exposed to actually writing SQL. He is unlikely to become an SQL expert, but using SQL behind a code facade, he should become one.

Re: Good system design

#115

Post by mexicocitinluez »

gethly wrote: post #66

Actually event-sourcing solves most of the pains - events, schema, push/pull, caching, distribution... whatever. The downside is that it is definitely not suitable for small projects and the overhead is substantial(especially during the development stage when you want to ship the product as soon as possible). On the other hand, once you get it going, it's an unstoppable beast.

There are some tools that try to solve this (MartenDb, for instance), but I wish there was an easier to way to integrate a system where some parts us ES and some parts don't.

Almost all the tools I've seen are either fully event-sourced or have nothing to do with event-sourcing. There aren't a ton of in-betweens.

Re: Good system design

#116

Post by [deleted] »

tossandthrow wrote: post #113
Yokohiii wrote: post #106

Earlier quoted context omitted.

I don't understand why all these problems should be easier handled with an ORM then with raw sql?

It is a granluarity tradeoff. With SQL you need to explicitly test all queries where the shape granularity is down to field level. When you map data onto an object model (in the dto sense, not oop sense) you have bigger building blocks. This gives a simpler application that is more reliable. Obviously you need to pick a performant orm - and it seems a lot of people in these threads have been traumatized. Personally,…

[deleted]

Re: Good system design

#117

Post by radlad »

tossandthrow wrote: post #113
Yokohiii wrote: post #106

Earlier quoted context omitted.

I don't understand why all these problems should be easier handled with an ORM then with raw sql?

It is a granluarity tradeoff. With SQL you need to explicitly test all queries where the shape granularity is down to field level. When you map data onto an object model (in the dto sense, not oop sense) you have bigger building blocks. This gives a simpler application that is more reliable. Obviously you need to pick a performant orm - and it seems a lot of people in these threads have been traumatized. Personally,…

The issue with GraphQL tends to be unoptimized joins instead. Is your GraphQL API available for public consumers? How do you manage them issuing inefficient queries?

I've most often seen this countered through data loaders (batched queries that are merged in code) instead of joins, or query whitelists.

Re: Good system design

#118

Post by scarface_74 »

bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

I have a very strict rule about any system I design or that I’m over against using stored procedures.

When I use to interview to be a developer at a company, it was always an automatic no for me if a company kept business logic in stored procedures and had a separate team of “database developers”.

As far as not doing joins in code, while I agree for the most part. GitHub itself has a rule against joining tables using sql that belong to different domains.

https://github.blog/engineering/infrastructure/partitioning-...

Re: Good system design

#119

Post by Yokohiii »

tialaramex wrote: post #95
bambax wrote: post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

Stored procedures seem like a win but the big problem is that while I could write the rest of the software in a very nice modern language like Rust, or more practically in C# since my team all know C# if I write a stored procedure it will be in Transact-SQL because that's the only choice. T-SQL was not a good programming language last century when it was vaguely current, and so no I do not want to write any significa…

I've only once tried to use stored procedures in mysql and it was almost impossible to debug back then. Very painful. Average devs already have issues being smart with their databases and stored procedures would add to that.

Stored procedures also add another risk. You have to keep them in sync with code, making releases more error prone. So you have to add extra layers of complexity to manage versioning.

I can see the advantage of extreme performance/efficiency gains, but it should be really big to be justified.

Re: Good system design

#120

Post by radlad »

dakiol wrote: post #94
mdavid626 wrote: post #78

Earlier quoted context omitted.

This is and “old-school” design. Nowadays I wouldn’t let apps meet in the database. Simple service oriented architecture is much preferred. Each app with its own data. Then such problems can be easily avoided.

It’s not old school, it’s actually solid design. I have worked too with people that think the frontend or even services should guide the design/architecture of the whole thing. Seems tempting and it has the initial impression that it works, but long terms it’s just bad design. Having Data structures (and mainly this means database structures) stable is key to long term maintenance.

> Seems tempting and it has the initial impression that it works, but long terms it’s just bad design.

This appears as an opinion rather than an argument. Could you explain what you find bad about the design?

In any case, I believe a DB per backend service isn't a decision driven by the frontend - rather, it's driven by data migration and data access requirements.

Post reply on HN