Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Address Sanitizer Internals

blog.gistre.epita.fr

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Address Sanitizer Internals

#11

Post by barco »

xtqctz wrote: post #2

This is great! I found these videos helpful, too: https://youtu.be/Tl1uZ7FBwFQ Does anyone know of a good explanation of HWAddress Sanitizer internals?

There are multiple versions of HWAsan.

One for ARMv8 with Top-Byte-Ignore: you can use the top byte of memory addresses to store a tag.

When you allocate memory you return the "tagged" pointer and internally store "this region has this tag".

When you dereference a pointer, you check that the tag matches what you expect in your internal data structure.

With memory tagging extensions you can do something similar but the checks are performed by the processor.

Re: Address Sanitizer Internals

#12

Post by turndown »

wyldfire wrote: post #8
yosefk wrote: post #5

Earlier quoted context omitted.

It comes up a lot in HN C++-related comment threads, for starters

I frequently bring up ASan on HN. it's a great way to mitigate C and C++'s shortcomings. But it's not a panacea and unlikely to be described that way here without swift rebuttal. C or C++ w/o ASan and UBSan is like skydiving w/o a parachute.

I wouldn’t say they explicitly mention ASan, but in general you will see certain well known C++ developers/community members insist that with a set of sanitizers you won’t have to worry about the kind of things safety focused programmers would like added to C++, all the time never mentioning false positives.

Re: Address Sanitizer Internals

#14

Post by searealist »

skobes wrote: post #9
searealist wrote: post #7

Earlier quoted context omitted.

I've never seen anyone claim that asan fully mitigates memory safety issues in C++. Perhaps you could link to one?

I've never seen that particular claim either, but I did previously believe that asan would reliably detect an out-of-bounds write if and when it occurs. So I learned something new from the OP (that this type of false negative is possible).

Yeah, I'm just responding to the guy who is probably the most internet-famous C++ hater in the world. I guess he likes to make up stuff too. The article is good.

Re: Address Sanitizer Internals

#15

Post by adrian_b »

skobes wrote: post #9
searealist wrote: post #7

Earlier quoted context omitted.

I've never seen anyone claim that asan fully mitigates memory safety issues in C++. Perhaps you could link to one?

I've never seen that particular claim either, but I did previously believe that asan would reliably detect an out-of-bounds write if and when it occurs. So I learned something new from the OP (that this type of false negative is possible).

ASAN, i.e. "-fsanitize=address" is a completely different and unrelated sanitizer than checking out-of-bounds accesses, like "-fsanitize=bounds-strict".

Checking out-of-bounds accesses must detect any out-of-bounds access done at run time. It is done by comparing the pointers or indices used for access with the array bounds.

(At least for gcc: "Initializers of variables with static storage are not instrumented.")

Out-of-bounds access checking and integer overflow detection should normally be enabled by default by any C/C++ developer, excluding only those functions where it has been determined experimentally that disabling the checks improves measurably the performance and which have been verified very carefully to prove that such exceptions cannot occur.

Unfortunately, in gcc and clang there is a large number of compilation options related to sanitizers. Most of them should always be enabled, to remove all problems created by the laxity of the C/C++ standards. Instead of listing on the command line the humongous number of such options, many of the most useful are included in "-fsanitize=undefined", but the compiler documentation must be studied to see what else may need to be added. At least for release builds, "-fsanitize-trap=all" is usually also desirable.

Re: Address Sanitizer Internals

#16

Post by adrian_b »

turndown wrote: post #12
wyldfire wrote: post #8

Earlier quoted context omitted.

I frequently bring up ASan on HN. it's a great way to mitigate C and C++'s shortcomings. But it's not a panacea and unlikely to be described that way here without swift rebuttal. C or C++ w/o ASan and UBSan is like skydiving w/o a parachute.

I wouldn’t say they explicitly mention ASan, but in general you will see certain well known C++ developers/community members insist that with a set of sanitizers you won’t have to worry about the kind of things safety focused programmers would like added to C++, all the time never mentioning false positives.

ASAN is only a probabilistic sanitizer, but adding deterministic checks, like out-of-bounds checks or integer overflow checks, is the same in C/C++ compiled with the appropriate options as what is done in any programming language where these checks are done by default.

In that case there are no false positives or negatives.

Re: Address Sanitizer Internals

#17

Post by adrian_b »

yosefk wrote: post #5
searealist wrote: post #4

Earlier quoted context omitted.

I've never seen anyone claim that.

It comes up a lot in HN C++-related comment threads, for starters

If it comes up, than that must be due to frequent confusions between ASAN and the many other completely different kinds of sanitizers, like the array bounds checker.

Unfortunately the documentation for the great number of sanitizers available in gcc and clang is both voluminous and incomplete.

For all of them their internals should have been very clearly documented, like in this article about ASAN.

Re: Address Sanitizer Internals

#18

Post by nneonneo »

adrian_b wrote: post #16
turndown wrote: post #12

Earlier quoted context omitted.

I wouldn’t say they explicitly mention ASan, but in general you will see certain well known C++ developers/community members insist that with a set of sanitizers you won’t have to worry about the kind of things safety focused programmers would like added to C++, all the time never mentioning false positives.

ASAN is only a probabilistic sanitizer, but adding deterministic checks, like out-of-bounds checks or integer overflow checks, is the same in C/C++ compiled with the appropriate options as what is done in any programming language where these checks are done by default. In that case there are no false positives or negatives.

Pray tell: what magic C/C++ compiler options do I add to enable deterministic OOB checks that never produce any false positives or negatives?

Re: Address Sanitizer Internals

#19

Post by shric »

"For this article, you’ll need the following knowledge:

Basic C understanding (Memory, Stack, Heap, Syscall)."

Obviously, since C doesn't prescribe any kind of heap, stack or syscall behavior (or if they even exist), I assume the author meant something like "Basic understanding of how C is often implemented on certain operating systems and hardware".

Re: Address Sanitizer Internals

#20

Post by nneonneo »

searealist wrote: post #4
yosefk wrote: post #3

One thing this explains is why ASan has false negatives. It's a great tool, but the typical comment that it fully mitigates memory safety issues is just not true (even assuming your tests actually trigger the memory safety bugs, which unlike eg code coverage there's no knowing if you achieved or not)

I've never seen anyone claim that.

Some examples from 1 minute with Algolia:

https://news.ycombinator.com/item?id=37479651

“All decent C compilers have compilation options so that at run-time any undefined actions, including integer overflow and out-of-bounds accesses, will be trapped.”

“Despite the hype, by default Rust is not safer than C compiled with the right options, because the default for Rust releases is also to omit many run-time checks.”

https://news.ycombinator.com/item?id=25922430

Upthread someone asks for a “-safe” flag which makes C a safe language. The reply is “It's called AddressSanitizer. You enable it with the compiler flag -fsanitize=address.” Several replies ensue pointing out how wrong this is.

Post reply on HN