The Law of Leaky Abstractions (2002)

No, reversible abstractions are just one kind of abstraction. For instance, a machine code sequence to a linear sequence of assembly instructions is a reversible abstraction. Not every machine code sequence is expressible as a linear sequence of assembly instructions, but every linear sequence of assembly instructions has a trivial correspondence to a machine code sequence.

However, consider the jump to a C-like language. The key abstraction provided there is the abstraction of infinite local variables. The compiler manages this through a stack, register allocation, and stack spilling to provide the abstraction and consumes your ability to control the registers directly to provide this abstraction. To interface at both levels simultaneously requires the leakage of the implementation details of the abstraction and careful interaction.

What you can do easily is what I call a separable abstraction, a abstraction that can be restricted to just the places it is needed/removed where unneeded. In certain cases in C code you need to do some specific assembly instruction, sequence, or even function. This can be easily done by writing a assembly function that interfaces with the C code via the C ABI. What is happening there is that the C code defines a interface allowing you to drop down or even exit the abstraction hierarchy for the duration of that function. The ease of doing so makes C highly separable and is part of the reason why it is so easy to call out to C, but you hardly ever see anybody calling out to say Java or Haskell.

Of course, that is just one of the many properties of abstractions that can make them easier to use, simpler, and more robust.

Abstractions work by restricting the domain of what you can do, then building on those restrictions. For example, raw hardware can jump anywhere, but structured programming constrains you to jump only to certain locations in order to implement if, for, functions, etc. It is precisely those restrictions that bring the benefits of structured programming; if you still frequently dipped into jumping around directly structured programming would fail to provide the guarantees it is supposed to provide. CRUD frameworks provide their power by restricting you to CRUD operations, then building on that. Immutable data is accomplished by forbidding you from updating values even though the hardware will happily do it. And so on.

Escape hatches under the abstractions are generally there precisely to break the abstractions, and break them they do.

Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. There's no way around it. Careful thought can make using an escape hatch less pain than it might otherwise be (such as the ORM that makes it virtually impossible to use SQL by successfully hiding everything about the SQL tables from you so you're basically typing table and column names by dead reckoning), but that's all that can be done.

One thing to do about this is that just as in the past few years the programming community has started to grapple with the fact that libraries aren't free but come with a certain cost that really adds up once you're pulling in a few thousand libraries for a framework's "hello world", abstractions that look really useful but whose restrictions don't match your needs need to be looked at a lot more closely.

I had something like that happen to me just this week. I needed a simple byte ring buffer. I looked in my language's repos for an existing one. I found them. But they were all super complicated, offering tons of features I didn't need, like being a writethrough buffer (which involved taking restrictions I didn't want), or where the simple task of trying to understand the API was quite literally on par with implementing one myself. So I just wrote the simple thing. (Aiding this decision is that broadly speaking if this buffer does fail or have a bug it's not terribly consequential, in my situation it's only for logging output and only effectively at a very high DEBUG level.) It wasn't worth the restrictions to build up stuff I didn't even want.

> Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to.

Native-code compilers commonly allow emitting assembly directly, but now your source code isn't portable between CPUs. Many interpreted languages, even most, allow FFI code to be imported, modifying the runtime accordingly, but now your program isn't portable between implementations of that language, and you have to be careful to make sure the behavior you've introduced doesn't mess with other parts of the system in unexpected ways.

Generalizing, it's often possible to drill down beneath the abstraction layer, but there's often an inherent price to be paid, whether it be taking pains to preserve the invariants of the abstraction, losing some of the benefits of it, or both.

There are better and worse versions of this layer, I would point to Lua as a language which is explicitly designed to cross the C/Lua boundary in both directions, and which did a good job of it. But nothing can change the fact that pure-Lua code simply won't segfault, but bring in userdata and it very easily can; the problems posed are inherent.

I loved this essay when it came out but I've come to dislike how "leaky abstraction" has become a form of low effort criticism that gets applied to almost anything.

> but it turns out in today's world, the abstraction of a reliable connection is TCP + a network administrator + a guy with wire snips + solder (metaphorically).

I think you’ve misunderstood the abstraction. In fact, TCP is not leaky because there’s wire snips or cable cuts. In fact, BGP will route around physical failures. But aside from that, it abstracts all the various failure modes as a single disconnection error. A leaky abstraction would be when you need to still distinguish the error type and TCP wouldn’t let you. A 100% reliable connection is physically impossible in any context (and an intrinsic concept of distributed systems which every abstraction is leaky over including the CPU bus) so if that’s your bar then all tech will be a leaky abstraction. It is at some level but not in a way that’s helpful to have a fruitful discussion.