41–50 of 71 posts
It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...
2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?
Earlier quoted context omitted.
2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?
Note this is for the virtual address space, not only for actual physical RAM. Having 256TB mmap’ed (to another kind of storage) does not seems totally out of reach.
iirc it was used in lisp compilers/interpreters to speed up object unboxing (as in answering the question: "what class does the object pointed by this pointer belong to?")
Earlier quoted context omitted.
Memory allocators are already returning more or less "random" pointers, so using pointers for sorting is already a bad idea. Even using pointers for equality-comparison isn't usually a good idea because the same memory location might be reused when the allocator recycles memory (those tags could actually help with making such "recycled" pointers unique over time).
Using pointers for sorting is fine as long as you don't care about the order (parent referenced std::set and it's not uncommon to use a set of pointers if you just want to store some objects). Likewise it's fine to use pointers for equality-comparisons as long as the pointer is valid (of course you have to dismiss the pointer when the memory is freed). I'm pretty sure this memory extension doesn't affect uniqueness o…
What I mean is for example: you get a pointer by allocating memory, then free that memory, then you allocate again and get a pointer to the same memory location. Without a tag those two pointers would be equal, even though they come from separate memory allocations. This can be a source of bugs if the pointer is also used as some sort of object id, and I know that at least I stumbled over this embarrassing problem more than once in C++ until I switched to tagged handles :)
isn't like, memory tagging explicitly forbidden in x86-64 ? iirc it was used in lisp compilers/interpreters to speed up object unboxing (as in answering the question: "what class does the object pointed by this pointer belong to?")
Of course, this doesn’t stop you from being able to[a], it just makes it harder.
[a]: The processor doesn’t know the “type” of a register’s value; It’s just bits. It could be a pointer, integer, floating point, etc. until you tell the processor to do something with it. So one could obviously use those extra bits to store data, then when a dereference is needed, store those bits in another register, dereference, then put the bits back. But that’s a lot of work to save a byte or two.
[0]: Intel SDM Volume 1 §3.3.7.1:
> In 64-bit mode, an address is considered to be in canonical form if address bits 63 through to the most-significant implemented bit by the microarchitecture are set to either all ones or all zeros.
isn't like, memory tagging explicitly forbidden in x86-64 ? iirc it was used in lisp compilers/interpreters to speed up object unboxing (as in answering the question: "what class does the object pointed by this pointer belong to?")
Emacs Lisp is an example; here's its tagging apparatus.
https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#...
Earlier quoted context omitted.
2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?
Note this is for the virtual address space, not only for actual physical RAM. Having 256TB mmap’ed (to another kind of storage) does not seems totally out of reach.
I can't imagine anyone needing more than 4PB in the next 30 years, and the security gains now is (in my opinion) worth the potential refactoring in 30+ years that would be required.
Not sure if this has been mentioned, but this exact technique (storing extra data in addresses) was used in the early days of the MacOS, with 24 bits for the address, and 8 bits for extra info (not sure if I can remember exactly what they were for). The 68000 only had 24 address lines after all. This allowed a maximum of 16Mb of RAM which was of course far more than would ever be in a computer... By the time the Mac…
The Mac IIci was the first 32 bit clean machine. Plenty of dynamically typed languages have used the bottom bits in an address to store tags. You have two available on a 32 bit machine, three on 64 bit. The SPARCv7 has instructions that will do arithmetic on tagged values.
Earlier quoted context omitted.
Using pointers for sorting is fine as long as you don't care about the order (parent referenced std::set and it's not uncommon to use a set of pointers if you just want to store some objects). Likewise it's fine to use pointers for equality-comparisons as long as the pointer is valid (of course you have to dismiss the pointer when the memory is freed). I'm pretty sure this memory extension doesn't affect uniqueness o…
> uniqueness of pointers... What I mean is for example: you get a pointer by allocating memory, then free that memory, then you allocate again and get a pointer to the same memory location. Without a tag those two pointers would be equal, even though they come from separate memory allocations. This can be a source of bugs if the pointer is also used as some sort of object id, and I know that at least I stumbled over…
How does this impact software that does memory pointer comparisons? Eg. Store pointers to all these objects into a std::set? The memory tag will now form part of the keys to this set, and would affect iteration order etc... It could cause breakage in existing applications.
Comparing pointers not coming from the same allocation is undefined behaviour per C++ standard.
Std::set uses it of course.