Live data from Hacker News

Good system design

seangoedecke.com

221–230 of 400 posts

Re: Good system design

#221
post #120
post #94

Earlier quoted context omitted.

It’s not old school, it’s actually solid design. I have worked too with people that think the frontend or even services should guide the design/architecture of the whole thing. Seems tempting and it has the initial impression that it works, but long terms it’s just bad design. Having Data structures (and mainly this means database structures) stable is key to long term maintenance.

> Seems tempting and it has the initial impression that it works, but long terms it’s just bad design. This appears as an opinion rather than an argument. Could you explain what you find bad about the design? In any case, I believe a DB per backend service isn't a decision driven by the frontend - rather, it's driven by data migration and data access requirements.

> In any case, I believe a DB per backend service isn't a decision driven by the frontend - rather, it's driven by data migration and data access requirements.

I think the idea of breaking up a shared enterprise DB into many distinct but communicating and dependent DB's was driven by a desire to reduce team+system dependencies to increase ability to change.

While the pro is valid and we make use of the idea sometimes when we design things, the cons are significant. Splitting up a DB that has data that is naturally shared by many departments in the business and by many modules/functional areas of the system increases complexity substantially.

In the shared model, when some critical attribute of an item (sku) is updated, then all of the different modules+functional areas of enterprise are immediately using that current and correct master value.

In the distributed model, there is significant complexity and effort to share this state across all areas. I've worked on systems designed this way and this issue frequently causes problems related to timing.

As with everything, no single solution is best for all situations. We only split this kind of shared state when the pros outweigh the cons, which is sometimes but not that often.

Re: Good system design

#222

> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…

This is awful advice. Simple and elegant design does not start with dismissing potential problems.

Those questions are all prompts to have a discussion in lieu of tech trivia hour. Those responses do not demonstrate wisdom, they reveal a lack of maturity. It's not the interviewers fault you refuse to be interviewed.

Re: Good system design

#223

Earlier quoted context omitted.

If you know that those are not the answers they are looking for, you can reasonably pass by modifying the answer only slightly, while still getting your point across. If you can't, you might be getting interviewed by people you do not what to work with and you should want to know that.

Except these are the people in your way of getting that job that could be potentially life/career changing for you financially or otherwise. In this market or depending on your situation that would be hard to ignore.

I think that's a red herring. You are a knowledge worker. You are paid to disagree when necessary. Yes, people will probably take offense when you say "that's just a dumb question" but if they can't at least be approached when you offer your opinion in a palatable(sic) way, that's simply not going to work.

Understand what is being asked. Your insight on a topic is being tested. Offer an answer that does not read like a dodge or a coin flip.

Re: Good system design

#224
post #173

What a great article. It's always a treat to read this sort of take. I have some remarks though. Taken from the article: > Avoid having five different services all write to the same table. Instead, have four of them send API requests (or emit events) to the first service, and keep the writing logic in that one service. This is not so cut-and-dry. The trade offs are far from obvious or acceptable. If the five services…

> And what exactly do you buy yourself? APIs can be evolved much more easily than shared database schemas. Having worked with many instances of each kind of system, I think this outweighs all of the other considerations, and I don't think I'll ever again design a system with multiple services accessing the same database schema. It was maybe a good idea if you were a small company in the early 2000s, when databases we…

Service specific views, my guy.

Re: Good system design

#225
post #14

> When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Oh yes! Never do a join in the application code! But also: use views! (and stored procedures if you can). A view is an abstraction about the…

Views are great. Stored procedures are cursed.

Re: Good system design

#226
post #141

Earlier quoted context omitted.

With an ORM your application code is your views. You can write reusable plain functions as abstractions, returning QuerySets that allow further filters being chained onto the query, before the actual SQL is materialized and sent to the database. The result of this doesn’t have to match the original object models you defined, it’s still possible to be flexible with group bys resulting in dictionaries.

Unpopular opinion. ORM by definition is the gcd of "supported databases" features. It exists only because people doesn't like the aesthetics of SQL but the cost to use them is immense.

Not unpopular. ORM hate is real. I like SQL Alchemy and Drizzle in projects for the features they give you for free (such as Alembic migrations and instant GraphQL server), but I still write SQL for most stuff.

Re: Good system design

#227

Earlier quoted context omitted.

I don't understand why all these problems should be easier handled with an ORM then with raw sql?

It is a granluarity tradeoff. With SQL you need to explicitly test all queries where the shape granularity is down to field level. When you map data onto an object model (in the dto sense, not oop sense) you have bigger building blocks. This gives a simpler application that is more reliable. Obviously you need to pick a performant orm - and it seems a lot of people in these threads have been traumatized. Personally,…

> 50 ms p99

You realize that’s abysmally bad performance for any reasonable OLTP query, right? Sub-msec (as measured by the DB, not including RTT etc.) is very achievable, even at scale. 2-3 msec for complex queries.

Re: Good system design

#228

Earlier quoted context omitted.

Stored procedures seem like a win but the big problem is that while I could write the rest of the software in a very nice modern language like Rust, or more practically in C# since my team all know C# if I write a stored procedure it will be in Transact-SQL because that's the only choice. T-SQL was not a good programming language last century when it was vaguely current, and so no I do not want to write any significa…

I've only once tried to use stored procedures in mysql and it was almost impossible to debug back then. Very painful. Average devs already have issues being smart with their databases and stored procedures would add to that. Stored procedures also add another risk. You have to keep them in sync with code, making releases more error prone. So you have to add extra layers of complexity to manage versioning. I can see t…

I'm a big postgres guy and in theory I love stored procedures (so many language options!) but you're 100% right that the downsides in terms of DX make them pretty much the last thing I reach for unless they're a big performance/simplicity win and I expect them to be pretty static over time.

Re: Good system design

#229

> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…

I’ve been in software for 20 years and it’s the first time I hear “back pressure”. Am I too old already?

Services, systems, and/or databases eventually provide back pressure when they fail or get overloaded. The idea is to design in back pressure to let the system degrade gracefully rather than fail chaotically.

Re: Good system design

#230
post #59
post #22

The distinction of stateful and stateless is one of the main criteria how we're dividing responsibilities between platform-infra and development. I know it's a bit untrue, but you can't do that many things wrong with a stateless application running in a container. And often the answer is "kill it and deploy it again". As long as you don't shred your dataset with a bad migration or some bad database code, most bad thi…

> but you can't do that many things wrong with a stateless application running in a container > As long as you don't shred your dataset with a bad migration or some bad database code, most bad things at this level can be fixed in a few minutes with a few redeployments. At some point between these statements you switched from stateless to stateful and I can't follow the rest of the argument.

If you mess up your application code in a stateless container, that's boring. Roll code back, and you're back where you want to be. This is stateless and easy.

If you introduce a migration like "UPDATE billing SET prices = 0 ; WHERE something This is still easy, you can catch it before it hits prod so you don't have to fix prod.

And prod could still be fixed if your database layer manages backups, just with a day or two of downtime. If you don't have backups, you may have permanently lost information, which could kill the company.

Post reply on HN