Linear Address Spaces: Unsafe at any speed (2022)

Is each branch-free run of instructions an object (which in general will be smaller than "function" or "method" objects) that can be abstracted? How does one manage locality ("these objects are the text of this function")?

Maybe one compromises and treats the text of a function as linear address space with small relative offsets. Of course, other issues will crop up. You can't treat code as an array, unless it's an array of the smallest word (bytes, say) even if the instructions are variable length. How do you construct all the pointer+capability values for the program's text statically? The linker would have to be able to do that...

If PHK, DJB, writes a post that comes across as clueless or flat out wrong, I'm going to read it and read it carefully. It does happen that says dumb and/or incorrect things from time to time, but most likely there will be a very cool nugget of truth in there.

Regarding what PHK seems to be asking for, I think it's... linear addressing of physical memory, yes (because what else can you do?) but with pointer values that have attached capabilities so that you can dispense with virtual to physical memory mapping (and a ton of MMU TLB and page table hardware and software complexity and slowness) and replace it with hardware capability verification. Because such pointers are inherently abstract, the fact that the underlying physical memory address space is linear is irrelevant and the memory model looks more like "every object is a segment" if you squint hard. Obviously we need to be able to address arrays of bytes, for example, so within an object you have linear addressing, and overall you have it too because physical memory is linear, but otherwise you have a fiction of a pile of objects some of which you have access to and some of which you don't.

I do not share the view that the unification of RAM and disk requires or entails linear regions of memory. In fact, the unification reduces the question of «do I have a contiguous block of size N to do X» to a mere «do I have enough memory to do X?», commits and rollbacks inclusive.

The issue of durability, however, remains a valid concern in either scenario, but the responsibility to ensure durability is delegated to the hardware.

Futhermore, commits and rollbacks are not sensitive to the memory linearity anyway; they are sensitive to durability of the operation, and they may be sensitive to the latency, although it is not a frequently occurring constraint. In the absence of a physical disk, commits/rollbacks can be implemented using the software transactional memory (STM) entirely in RAM and today – see the relevant Haskell library and the white paper on STM.

Lastly, when everything is an object in the system, the way the objects communicate with each other also changes from the traditional model of memory sharing to message passing, transactional outboxes, and similar, where the objects encapsulate the internal state without allowing other objects to access it – courtesy of the object-oriented address space protection, which is what the conversation initially started from.

> Why do we even have linear physical and virtual addresses in the first place, when pretty much everything today is object-oriented?

What a weird question, conflating one thing with the other.

I’m working on a object capability system, and trying hard to see if I can make it work using a linear address space so I don’t have to waste two or three pages per “process” [1][2] I really don’t see how objects have anything to do with virtual memory and memory isolation, as they are a higher abstraction. These objects have to live somewhere, unless the author is proposing a system without the classical model of addressable RAM.

—-

1: the reason I prefer a linear address space is that I want to run millions of actors/capabilities on a machine, and the latency and memory usage of switching address space and registers become really onerous. Also, I am really curious to see how ridiculously fast modern CPUs are when you’re not thrashing the TLB every millisecond or so.

2: in my case I let system processes/capabilities written in C run in linear address space where security isn’t a concern, and user space in a RISC-V VM so they can’t escape. The dream is that CHERI actually goes into production and user space can run on hardware, but that’s a big if.

The memory management story is still a big question: how do you do allocations in a linear address space? If you give out pages, there’s a lot of wastage. The alternative is a global memory allocator, which I am really not keen on. Still figuring out as I go.

Look behind the curtains, and the space for improvement over the UNIX model are enormous. Our only saving grace is that computers have gotten ridiculously fast.