21–30 of 43 posts
1. A lot of it is heavily dependent on implementation details. (Yeah we have to care about XID, but we shouldn’t have to care about it exponentially for nested scope workloads once that scope exits.)
2. Actual nested transactions could help alleviate that without too much burden on the MVCC implementation. (This would provide a mechanism for isolating and cleaning up scoped XIDs.)
When doing multiple steps that need to track external state, subtransactions can greatly simplify the schema you need as well as make the database interactions more efficient. You need to safely record progress at each step, and you can't afford to lose previous steps because a transaction gets rolled back unexpectedly. You could start a transaction and commit between each step, but this gets pretty annoying if you need to SELECT FOR UPDATE or otherwise make sure what you are doing is concurrency safe, since this would require you to track all the resources you need as you go, which would greatly complicate the design of your non-sql code. Not to mention the added failure modes of not being able to reacquire those resources after step N for whatever reason.
Where things get stupid is ORMs and frameworks that start sub transactions all over the place when you don't need them and aren't even trying to use them. I'm looking at you, Django transaction.atomic(), you absolute idiot.
If the subtransaction is later rolled back, this is starting to sound like the kind of speculative execution information leak that led to the Spectre attack. I wonder if you could base a similar attack on this?
Using subtransactions appropriately is unlikely to trigger any noticeable performance differences, any more than 'having less transactions' overall will. When doing multiple steps that need to track external state, subtransactions can greatly simplify the schema you need as well as make the database interactions more efficient. You need to safely record progress at each step, and you can't afford to lose previous ste…
cockroachdb implements subtransactions and I don't think any of these issues apply to cockroach's implementation.
> ...we do not recommend their use in new applications.
> The reality is that nested transactions are a product of the early days of software engineering, in the 1990s, back when systems were tightly coupled and the Internet and the Cloud were not yet very relevant. ...
> Nowadays, such tight coupling has a bad rep. This is because two additional decades of software engineering have taught us that implicit global state really, really does not play well with distributed services where some components may fail even as part of normal load ...
> Additionally, nested transactions can amplify performance anomalies ...
> Finally, nested transactions can run afoul of correctness in distributed apps. In fact, the idea of multi-component transactions in client code really evokes the idea of a bull in a china shop. As long as all is well and the transaction is due to commit, the idea somewhat makes sense. However, what happens when the database (and not the client) decides the transaction is un-committable and must be aborted, for example because of a serializability conflict or a node restarting for maintenance? It is not just the database state that must be rolled back; all the possible side effects performed by the components holding the transaction must also be rolled back.
They implemented it, but do not recommend using it.
Why is the XID a 32bit number? Surely we can move to 64bit and punt the wraparound issue a bit?
That said, I'm not sure the concern holds water. At 2bil rows we're talking 7.4GiB of disk vs 15GiB. Depending on what % of your database you want to keep in memory, maybe it goes ok.
Earlier quoted context omitted.
I've seen plenty over the years as a professional developer of 20+ years and they are opinion pieces for the most part. I'd rather see a title along the lines of "Why you should avoid subtransactions". Drinking bleach is harmful.
The original was Dijkstra's "Go To Statement Considered Harmful". Many considered that title unnecessarily provocative, but it ended up creating a new CS/programming meme. Now, I see "Considered Harmful" as just a concise way of alluding to that style of article, basically saying "here are the not-well-known downsides of a commonly-used thing". Many "Considered Harmful" articles do not live up to that promise, but I…
Earlier quoted context omitted.
I'm the co-author of one of such ones: "Schema Later Considered Harmful" [1]. I don't see what's wrong with this "type titles", may you elaborate? :) [1] https://www.enterprisedb.com/blog/schema-later-considered-ha...
They're hyperbolic and don't actually give you any useful information on why they're "harmful" or what's wrong with the subject being discussed.