Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Arm64 memory tagging extension in Linux

lwn.net

Post reply on HN
Print view View original

21–30 of 71 posts

Re: The Arm64 memory tagging extension in Linux

#21

Post by jmgao »

sherincall wrote: post #17
londons_explore wrote: post #7

It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...

The estimated number of atoms on earth is on the order of 1e50, which is 60 bits. If you take the 4 bits out of the 64bit address for tagging, you can still address individual atoms. EDIT: My math is bad, see comments below. I can unsarcastically say that no one will ever need full 64bit addresses on this planet.

10^50 is definitely not 60 bits: log_2(10^50) = 50 * log_2(10), and log_2(10) > 3

Re: The Arm64 memory tagging extension in Linux

#22

Post by sherincall »

five-robert wrote: post #19
sherincall wrote: post #17

Earlier quoted context omitted.

The estimated number of atoms on earth is on the order of 1e50, which is 60 bits. If you take the 4 bits out of the 64bit address for tagging, you can still address individual atoms. EDIT: My math is bad, see comments below. I can unsarcastically say that no one will ever need full 64bit addresses on this planet.

2^66 bits - estimated storage space at Google data warehouse as of 2013 [0] 2^71 bits - total hard drive capacity shipped in 2016 [0] I doubt your calculations. [0] https://en.wikipedia.org/wiki/Orders_of_magnitude_(data)

Yes, you're totally right, my math is way off.

Re: The Arm64 memory tagging extension in Linux

#23

Post by throwaway2048 »

five-robert wrote: post #16
londons_explore wrote: post #7

It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...

2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?

It ought to be enough for anyone

Re: The Arm64 memory tagging extension in Linux

#24

Post by pm215 »

sherincall wrote: post #18
flohofwoe wrote: post #13

Earlier quoted context omitted.

Memory allocators are already returning more or less "random" pointers, so using pointers for sorting is already a bad idea. Even using pointers for equality-comparison isn't usually a good idea because the same memory location might be reused when the allocator recycles memory (those tags could actually help with making such "recycled" pointers unique over time).

Be that as it may, existing software that used to work will no longer work, and the developers might not be around/able to fix it. There are also cases of custom suballocators or arrays of objects - Looking at an address makes it possibly to figure out which array it belongs to. This code would break. Granted, it would still be possible to do all this if you just mask off the tag bits, but it requires a software chan…

The "suballocate out of some arrays" code should not break, because the whole array would be allocated at once and so would have the same tag for the whole range. Code that does a simple "is this pointer value inside the "array_base + size" range" continues to work, because array_base has whatever tag malloc() handed out for that array, and so do the pointer values that the suballocator handed out. I think for MTE to break your code you would have to be doing some pretty weird stuff with pointer arithmetic (beyond just the usual "technically maybe this is undefined behaviour but it works" level stuff).

It's always the case that some software that does things that are not valid-by-the-language-standard might break if run on a newer version of the OS or a newer system library version (remember the big flap about glibc memcpy() changing its behaviour when called for overlapping regions?). You don't want to break lots of software gratuitously, but sometimes the tradeoff is worth making.

Re: The Arm64 memory tagging extension in Linux

#25

Post by [deleted] »

londons_explore wrote: post #4

Random read and write of a big memory region could now presumably cause double the number of DRAM accesses, and half the performance, since the tag mapping will need to be read in addition to the actual data. If the read of a memory address could have side effects (as is common with memory mapped hardware), does the hardware give guarantees that the tag will be checked before the memory is read or written? If so, tha…

[deleted]

Re: The Arm64 memory tagging extension in Linux

#26

Post by jackewiehose »

flohofwoe wrote: post #13
londons_explore wrote: post #2

How does this impact software that does memory pointer comparisons? Eg. Store pointers to all these objects into a std::set? The memory tag will now form part of the keys to this set, and would affect iteration order etc... It could cause breakage in existing applications.

Memory allocators are already returning more or less "random" pointers, so using pointers for sorting is already a bad idea. Even using pointers for equality-comparison isn't usually a good idea because the same memory location might be reused when the allocator recycles memory (those tags could actually help with making such "recycled" pointers unique over time).

Using pointers for sorting is fine as long as you don't care about the order (parent referenced std::set and it's not uncommon to use a set of pointers if you just want to store some objects). Likewise it's fine to use pointers for equality-comparisons as long as the pointer is valid (of course you have to dismiss the pointer when the memory is freed).

I'm pretty sure this memory extension doesn't affect uniqueness of pointers... that would indeed break a lot of software ;-)

Re: The Arm64 memory tagging extension in Linux

#27

Post by tpxl »

five-robert wrote: post #16
londons_explore wrote: post #7

It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...

2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?

I am flirting with the idea of having 1TB of RAM on my servers as a small start-up. Having 256TB in 20 years seems reasonable.

Re: The Arm64 memory tagging extension in Linux

#28

Post by nnx »

five-robert wrote: post #16
londons_explore wrote: post #7

It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...

2^48 == 256TB. Anyone needs more than 256 Terabytes of RAM?

Note this is for the virtual address space, not only for actual physical RAM. Having 256TB mmap’ed (to another kind of storage) does not seems totally out of reach.

Re: The Arm64 memory tagging extension in Linux

#29

Post by cesaref »

Not sure if this has been mentioned, but this exact technique (storing extra data in addresses) was used in the early days of the MacOS, with 24 bits for the address, and 8 bits for extra info (not sure if I can remember exactly what they were for). The 68000 only had 24 address lines after all. This allowed a maximum of 16Mb of RAM which was of course far more than would ever be in a computer...

By the time the Mac II came out (68020 with 32 bit address bus), it was apparent that this was going to be an issue, and there was an effort to remove this logic. Those early machines were a combination of ROM and loaded OSes, so there was an effort to clean the code for both the distributed OS and the stuff included in the ROMs of the machines.

I think if memory serves the Mac IIcx from around '89 or so was the first machine with 32 bit clean roms, and with the right OS version you were good for >16mb of RAM.

I've also got a hazy memory of some architecture (i'm going to guess DEC Alpha) which only read from 32 bit boundaries, and where the bottom 2 bits of the address were masked out, so you could store data there too...

Anyhow, the real take home from all that was that misusing addresses to store additional information, which might seem like a clever trick is likely to bite you at some point in the future.

Re: The Arm64 memory tagging extension in Linux

#30

Post by jpcooper »

pm215 wrote: post #20
jpcooper wrote: post #10

So a key is stored in the unused portion of memory pointers which should match the key associated to the memory it points to. I couldn’t figure out from the article where the key is stored in the corresponding memory. Can anyone explain?

mwsealey's comment on the LWN article (in response to somebody asking that question there) has a good explanation of it. Basically exactly where the keys are stored is up to the implementation, but one reasonable implementation is for the memory controller to put them in the top part of the system DRAM (which is then not made available as 'normal' RAM). Architecturally the allocation tag space is entirely separate fr…

Thanks for the explanation.
Post reply on HN