Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChkTag: x86 Memory Safety

community.intel.com

Post reply on HN
Print view View original

111–120 of 147 posts

Re: ChkTag: x86 Memory Safety

#111

Post by sparkie »

monocasa wrote: post #17
sparkie wrote: post #2

Sparse on details. Presumably will be based on the existing Linear Address Masking/Upper Address Ignore specs, which are equivalent, and will be similar to CHERI. If so it needs to be opt-in or at least opt-out per process, because many language runtimes use these pointers bits to optimize dynamic types, and would suffer a big performance hit if they were unable to use them.

Not a whole lot of language runtimes (if any) really depend on upper address ignore. AFAIK, AMD only added it in Zen4.

You don't need the hardware UAI/LAM to make use of the high pointer bits. The most common technique is to use `shl reg, 16; sar reg, 16`, which will shift in ones or zeros from the left depending on the 47th bit.

Several runtimes use high bits tagging combined with NaN-boxing, and have been doing so since before LAM/UAI existed.

Re: ChkTag: x86 Memory Safety

#112

Post by sparkie »

wbl wrote: post #20
sparkie wrote: post #2

Sparse on details. Presumably will be based on the existing Linear Address Masking/Upper Address Ignore specs, which are equivalent, and will be similar to CHERI. If so it needs to be opt-in or at least opt-out per process, because many language runtimes use these pointers bits to optimize dynamic types, and would suffer a big performance hit if they were unable to use them.

Dynamic types have classically used the lower bits freed by alignment constraints. If I know a cons cell is 16 bytes then I can use the low 4 bits of an address to store enough type info to disambiguate.

There's numerous techniques used. Many are covered in Gudeman's 1993 paper "Representing Type Information in Dynamically Typed Languages"[1], which includes low-bits tagging, high-bits tagging, and NaN-boxing.

The high bits let us tag more types, and can be used in conjunction with low bits tagging. Eg, we might use the low bits for GC marking.

[1]:https://web.archive.org/web/20170705085007/ftp://ftp.cs.indi...

Re: ChkTag: x86 Memory Safety

#113

Post by sparkie »

secondcoming wrote: post #42
themafia wrote: post #24

Earlier quoted context omitted.

Is it a guarantee that a 16 byte object would be 16 byte aligned?

In C++ you can force that with alignas(), I would imagine other low level languages offer something similar. If you're using a custom allocator you'd have to enfore it yourself which should be fine since you have full control. https://en.cppreference.com/w/cpp/language/alignas.html

C23 also has `alignas` and `alignof` (`_Alignas`/`_Alignof` in C11 with the lowercase as macros in stdalign.h), and also provides `aligned_alloc` and `free_aligned_size` in stdlib.

Re: ChkTag: x86 Memory Safety

#114

Post by sparkie »

pansa2 wrote: post #68
fooker wrote: post #35

Earlier quoted context omitted.

Lua does

I thought one of Lua’s selling points is that it’s written in highly standard-compliant C code. I wouldn’t expect it to do anything non-portable like bitwise manipulation of pointers?

It's fine to use techniques like this in standard C provided you guard them with the preprocessor and provide a fallback option on unsupported hardware.

Re: ChkTag: x86 Memory Safety

#115

Post by jandrewrogers »

sparkie wrote: post #110
jandrewrogers wrote: post #99

Earlier quoted context omitted.

If you restrict yourself to all variants of x86 and ARM, the number of high bits for which I could not find conflicting uses is 6 bits (bits 57-62). The other high bits are reserved in some hardware contexts and therefore may create conflicts. Using 16 bits may be risky on recent x86. For example, IIRC Linux enables 5-level page tables on microarchitectures that support it, which can put valid address data in bits 48…

> Using 16 bits may be risky on recent x86. For example, IIRC Linux enables 5-level page tables on microarchitectures that support it, which can put valid address data in bits 48-56. Linux will not allocate past the 47-bit range, even with 5-level paging enabled, unless specifically requested, by providing a pointer hint to `mmap` with a higher address. https://www.kernel.org/doc/html/v5.14/x86/x86_64/5level-pagi...

Ah, thanks for the detail! I was unaware that this was how it worked.

Re: ChkTag: x86 Memory Safety

#117

Post by Panzerschrek »

It would be nice to know how this memory safety instructions should be used by software developers. Assuming I write C++ code, what should I do? Enable some new compiler flags? Use special runtime library? Use some special variant of the language standard library which uses these new instructions? Completely rewrite my code to make it safe?

Re: ChkTag: x86 Memory Safety

#118

Post by pjmlp »

gpm wrote: post #67

It seems very strange to me to finally get around to this right as we are finally getting low level software that no longer needs it (and we've had high level software that doesn't need it for ages). At this point I think I'd prefer the transistor budget and bits of memory were spent on other things.

We have had it before C was even invented, Burroughs nowadays still sold as Unisys ClearPath MCP was written in ESPOL, latter NEWP, with zero Assembly.

The compiler provides intrisics, has bounds checking for strings and arrays.

PL/I and its variants were also used across several systems, as were ALGOL dialects.

Note C.A.R Hoare Turing award speech in 1980,

"A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."

The "1980 language designers and users have not learned this lesson" is meant to be C without explicit refering to it.

Re: ChkTag: x86 Memory Safety

#119

Post by pjmlp »

georgeburdell wrote: post #3

I wonder what happened that Apple/ARM has implemented something similar at nearly the same time. https://security.apple.com/blog/memory-integrity-enforcement...

The very first commercial system of this was done by Oracle in Solaris SPARC since 2015.

https://docs.oracle.com/en/operating-systems/solaris/oracle-...

Intel had a first attempt at this with MPX, but the design had flaws,

https://en.wikipedia.org/wiki/Intel_MPX

Then there is CHERI and the related ARM Morello,

https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

Apple isn't a first here, one first among general public.

Re: ChkTag: x86 Memory Safety

#120

Post by pjmlp »

raverbashing wrote: post #61

It's ok the C committee will make sure to fumble this up even with HW support

I don't have any hopes WG14 will ever care to improve C's safety beyond what is possible manually writting Assembly code, it is even worse because Assembly doesn't have time travel with UB.
Post reply on HN