331–340 of 400 posts
If the author meant “dictionary” in a sense of a hash map, that’s not quite correct. In relational databases, indexes are usually B-trees, which are ordered, unlike hash maps. A B-tree can help with range-searches, ORDER BY and even merge joins, not just equality-searches.
> But in most cases replication lag can be worked around with simple tricks: for instance, when you update a record but need to use it right after, you can fill in the updated details in-memory instead of immediately re-reading after a write. I get it, but that sounds very finicky code to get right and a good source of hard-to-debug bugs.
1. Essentially every RDBMS except MySQL has a RETURNING clause, so you can read your write essentially for free (and even MySQL lets you read the auto-increment value it inserted).
2. Barring that, how is this finicky to get right? If you get an ack back from the DB, assuming you haven’t done silly things to fsync settings, it’s written. It’s durable. So, within the same try/except or similar, take what you just told it to write, and use it. If you don’t get an ack back, it did not write, so don’t use what you had stored.
>You have two options: fail open and let the request through, or fail closed and block the request with a 429. If the metaphor of a software circuit breaker is meant to emulate an electrical circuit breaker, then it seems to me that these two are inverted. Whenever a physical circuit breaker is open, it is not dangerous and not passing current.
Earlier quoted context omitted.
> Additionally, having five services accessing the same database is a code smell. Counterpoint (assuming by database you mean database cluster, not a schema): having a separate physical DB for each service means that for most places, your reliability has now gone from N to N^M.
From which perspective? If a service is up, but is unable to do anything since another service is down, what good does it do other than increase some metrics on some dashboard. (Note that we are specifically talking about coupled services since the implication is writing to a single db being split up into multiple dbs - a distributed monolith).
Earlier quoted context omitted.
If a candidate doesn’t ask clarifying questions that lead them to an understanding of QPS, storage requirements, and throughput considerations, that’s a mark against. At that point, if you want to see them design a distributed system with all the bells and whistles, you should stop them, tell them the kind of traffic they need to handle, then let them go again. If they persist in designing a system that cannot handle…
The problem with this is people seem to have mismatched understandings of what a single system can handle. e.g. my 8 year old quad core i5 desktop with a bit of batching optimization can handle 5 digit requests per second with 15 ms p99 with some nontrivial application logic doing several joins. I don't think I've tried that same benchmark on a modern minipc, but I expect it should be similar. That's well above what…
Like when I inform teams complaining of “slow queries” that the DB is executing them in sub-msec time. No idea what the rest of your stack is doing, but good luck with figuring that out - it ain’t me.
Earlier quoted context omitted.
> Plus the maintenance overhead that migrations of such shared tables come with. Moving your data types from SQL into another language solves exactly 0 migration problems. Every migration you can hide with that abstraction language you can also hide in SQL. Databases can express exactly the same behaviors as your application code.
I’m generally pro SQL-as-interface, but this is just wrong. Not only are there all sorts of bizarre constraints imposed by databases on migration behavior that application code can’t express (for example, how can I implement a transaction-plus-double-write pattern to migrate to use a new table because the locks taken to add an index to the old table require unacceptably long downtime? There are probably some SQL engi…
If for some reason you do need to migrate data to a new table, triggers.
If somehow these still don’t solve your problem, ProxySQL or the equivalent (you are running some kind of connection pooler, right?) can rewrite queries on the fly to do whatever you want.
Earlier quoted context omitted.
> Plus the maintenance overhead that migrations of such shared tables come with. Moving your data types from SQL into another language solves exactly 0 migration problems. Every migration you can hide with that abstraction language you can also hide in SQL. Databases can express exactly the same behaviors as your application code.
I’m generally pro SQL-as-interface, but this is just wrong. Not only are there all sorts of bizarre constraints imposed by databases on migration behavior that application code can’t express (for example, how can I implement a transaction-plus-double-write pattern to migrate to use a new table because the locks taken to add an index to the old table require unacceptably long downtime? There are probably some SQL engi…
You can take it one step further and implement the “uniform service layer” in the database itself - using stored procedures and views.
This has downsides, like strong coupling with the specific DBMS, and difficulty of development in a comparatively primitive SQL dialect, but protects the database from “naughty” clients and can have tremendous performance advantages in some cases.
Earlier quoted context omitted.
> In any case, I believe a DB per backend service isn't a decision driven by the frontend - rather, it's driven by data migration and data access requirements. I think the idea of breaking up a shared enterprise DB into many distinct but communicating and dependent DB's was driven by a desire to reduce team+system dependencies to increase ability to change. While the pro is valid and we make use of the idea sometimes…
I disagree. I generally understand the problem a "split-up" database brings to the table. This is how people designed things in the last many decades. What I propose is to leave this design behind. The split up design fits modern use cases much better. People want all kind of data. They want to change what data they want rather often. "One" database for all of this doesn't really work -- you can't change the schema s…
DB schema is supposed to be inflexible and strict; that’s how you can guarantee that the data it’s storing is correct.
> The 90s are over
And now we have a generation of devs who think that 1 msec latency for disk reads is normal, that applications need to ship their own OS to run, and that SQL is an antiquated language that they don’t need to bother to learn.
> I’m often alone on this. Engineers look at complex systems with many interesting parts and think “wow, a lot of system design is happening here!” In fact, a complex system usually reflects an absence of good design. For any job-hunters, it's important you forget this during interviews. In the past I've made the mistake of trying to convey this in system design interviews. Some hypothetical startup app > Interviewer…
> SQL and NoSQL don’t matter much
Database is literally the most important architectural decision possible, next to the application programming language.
(Prove me wrong)