The Law of Leaky Abstractions (2002)

I've long been having a hunch that we're currently in the "wild west of abstraction".

I think we're missing an essential constraint on the way we do abstraction.

My hunch is that this constraint should be that abstractions must be reversible.

Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. You could in theory of course modify the compiler output after it's been generated, but then you'd have to somehow manually keep that work in sync whenever you want to re-generate. Using an abstraction kinda locks you into that layer.

I see this problem appearing everywhere:

- Use framework Write from scratch

- Use an ORM Write raw SQL

- Garbage collection Manual memory management

- Using a DSL Writing raw language code

- Cross platform UI framework Native UI code

- ...

I think we're missing a fundamental primitive of abstraction that allows us to work on each layer of abstraction without being locked in.

If you have any thoughts at all on this, please share them here!

I like the idea of TCP as a leaky abstraction because it points out the difficulty of engineering the abstraction we really want. It would be wonderful for TCP to be a guaranteed connection abstraction, but it turns out in today's world, the abstraction of a reliable connection is TCP + a network administrator + a guy with wire snips + solder (metaphorically). Maybe down the road, AIs and repair bots will be involved, and the guaranteed connection abstraction might become real or much much stronger. Although it gets more complicated because if a message takes hours to deliver, is that going to work for your application? Yes if you're archiving documents, no if you're trying to set up a video conference call or display a web page.

TCP is problematic in modern circumstances (think: Inside a data center) because a response within milliseconds is what's expected to make the process viable. TCP was designed to accommodate some element of the path being a 300 Baud modem, where a response time in seconds is possible as the modem dials the next hop, so the TCP timeouts are unuseable. QUIC was developed to address this kind of problem. My point being, the abstraction of a guaranteed _timely_ connection is even harder.

I think Joel could have expanded his thoughts to include the degree of leak. SQL is a leaky abstraction itself, yes, but my own take is that ORMs are much leakier: Every ORM introduction document I've read explains the notation by saying "here's the sql that is produced". I think of ORMs as not a bucket with holes, but a bucket with half the bottom removed.

Joel of course hits the nail on the head about the two major things that cause abstractions to fall apart: performance and bugs (or debugging). In programming languages we talk about abstractions all the time--PL of course is all about abstractions. A computational abstraction like a bytecode, source language, or even machine code, can be proven be a proper (or full) abstraction, meaning there is no way for implementation details to leak in--you cannot observe the electrons flowing by executing A+B, after all.

...until you start measuring sidechannels, or the CPU or compiler has a bug.

I think about this a lot when dealing with VMs; a complex VM cannot hide its complexity when programs care about execution time, or when the VM actually has a bug.

That's not really true of, at least C compilers. Because compilers have ABI's and fixed calling conventions, it's straightforward, documented, and not uncommon (depending on your application area/deployment target) to drop down to the ASM layer if you need to do that.

It's definitely one of those things that makes C nice for bare metal programming.