Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

gcc.gnu.org

Post reply on HN
Print view View original

11–20 of 28 posts

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#11

Post by beagle3 »

sparky wrote: post #7
beagle3 wrote: post #4

Earlier quoted context omitted.

Do you have experience with valgrind? Can you compare & contrast the two? Also, I thought that the compiler additions was implemented years ago, several times (tristan gingold's "checker-gcc" first, then the bounds-checking patches and most recently "-fmudflap"), so that asan's functionality would basically only require rewriting libmudflap - but it apparently requires a much deeper surgery of gcc. Can anyone familia…

This page is a good high-level summary: http://code.google.com/p/address-sanitizer/wiki/ComparisonOf... . I've used most of the tools on that page, including Valgrind. Valgrind interprets your program, and hooks loads, stores, malloc/free, etc. to track memory usage. ASan is a compiler pass that inserts extra instructions around loads and stores to drive per-address state machines that live in a shadow area of memory…

Thanks! That is very helpful. I'm looking forward to trying on the stable gcc

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#12

Post by chubot »

hayksaakian wrote: post #6

As a person unfamiliar with the concepts presented in the OP, I would not mind if someone explained its significance in layman's terms.

Not sure how "layman" to get... but here you go :)

ASAN helps find difficult-to-reproduce bugs, e.g. data races. A data race occurs when two or more threads access the same memory locations in a undefined order. The OS can interrupt threads at any time, it's hard to test all possible interleavings of instructions from multiple threads. Programmers typically have some invariants in their mind but it's really easy to make a mistake, and hard to detect it.

To use the tool, you compile a C/C++ program in a different way. The ASAN toolchain instruments your memory accesses. Then you run your program (or unit tests for library code). And it will tell you if there were memory errors like data races. Then you look at your source code and fix the bug.

Valgrind does a similar thing and is a standard open source tool. As mentioned on the page, this is faster.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#13

Post by mrich »

I didn't know it is that close to becoming part of GCC, that's great!

I have been trying clang+ASan on a larger project with mixed success. While it could catch a known bug, it seemed to miscompile some essential startup code and therefore I could not get it running in regression tests. Never got around to debugging it, but now I can try again with GCC.

There is also another interesting project out there, although it is not as far along as ASAN:

http://safecode.cs.illinois.edu/

This project aims to track exact memory bounds of all objects. ASan will not detect out of bounds accesses that go to allocated memory of another object, but SAFECode would catch that.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#15

Post by mrich »

chubot wrote: post #12
hayksaakian wrote: post #6

As a person unfamiliar with the concepts presented in the OP, I would not mind if someone explained its significance in layman's terms.

Not sure how "layman" to get... but here you go :) ASAN helps find difficult-to-reproduce bugs, e.g. data races. A data race occurs when two or more threads access the same memory locations in a undefined order. The OS can interrupt threads at any time, it's hard to test all possible interleavings of instructions from multiple threads. Programmers typically have some invariants in their mind but it's really easy to m…

What you described is Thread Sanitizer (clang toolchain). The corresponding valgrind-based tools would be Helgrind and DRD.

ASan can only catch memory errors in C/C++ programs. A memory error is an access outside the allocated memory of an object, e.g. due to incorrect pointer arithmetics. Such errors would otherwise go undetected but can cause all kinds of errors in the program, like corrupted data, crashes etc.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#16

Post by DannyBee »

jameswilsterman wrote: post #14

Speaking of which, does anyone know of any good USPS address sanitizer / matching software?

Depends on what your goal is. The USPS has a number of products that do this kind of thing (See https://ribbs.usps.gov under Address Quality Services).

So if you are looking for something specifically to clean up or test software that cleans up USPS addresses, there you go.

If you want something that takes random address-looking things and tries to find a USPS address that matches it, that's harder :)

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#17

Post by nnethercote »

sparky wrote: post #7
beagle3 wrote: post #4

Earlier quoted context omitted.

Do you have experience with valgrind? Can you compare & contrast the two? Also, I thought that the compiler additions was implemented years ago, several times (tristan gingold's "checker-gcc" first, then the bounds-checking patches and most recently "-fmudflap"), so that asan's functionality would basically only require rewriting libmudflap - but it apparently requires a much deeper surgery of gcc. Can anyone familia…

This page is a good high-level summary: http://code.google.com/p/address-sanitizer/wiki/ComparisonOf... . I've used most of the tools on that page, including Valgrind. Valgrind interprets your program, and hooks loads, stores, malloc/free, etc. to track memory usage. ASan is a compiler pass that inserts extra instructions around loads and stores to drive per-address state machines that live in a shadow area of memory…

> Valgrind interprets your program

No, it has a JIT. It's running compiled code. See http://valgrind.org/docs/valgrind2007.pdf for details, section 3 in particular.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#18

Post by dllthomas »

xyzzy123 wrote: post #8
dllthomas wrote: post #3

"Compile-time valgrind" seems incorrect. It is compile-time addition of run-time valgrind-like instrumentation.

Yes, that's right. It does do the same thing as valgrind but the practical benefit of compile-time instrumentation rather than dynamic translation is that ASAN is enormously faster. A lot of people are using this on their fuzzing rigs with large software applications like Firefox. ASAN hugely decreases cost per test cycle (and therefore cost per bug, without changing fuzzers). A friend and I chipped in to get a fuzz…

Don't get me wrong; I don't mean to say it's not fantastic, just that the headline was misleading. My expectations for "Hey, we managed to move this stuff to compile time" is that it will be 1) more interesting theoretically, and 2) less interesting practically (at least in the short term). Either can be fantastic, and these expectations are sometimes violated besides, but I just wanted to give a heads up to others (at least, those that skim the HN comments first) or be corrected if my reading of it was wrong.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#19

Post by sparky »

nnethercote wrote: post #17
sparky wrote: post #7

Earlier quoted context omitted.

This page is a good high-level summary: http://code.google.com/p/address-sanitizer/wiki/ComparisonOf... . I've used most of the tools on that page, including Valgrind. Valgrind interprets your program, and hooks loads, stores, malloc/free, etc. to track memory usage. ASan is a compiler pass that inserts extra instructions around loads and stores to drive per-address state machines that live in a shadow area of memory…

> Valgrind interprets your program No, it has a JIT. It's running compiled code. See http://valgrind.org/docs/valgrind2007.pdf for details, section 3 in particular.

You're right (and you should know!), my mistake. I knew it too, and just mistyped. I would edit the post above if I could.

Re: Google Address Sanitizer ("compile-time valgrind") to be part of GCC 4.8

#20

Post by sparky »

mrich wrote: post #15
chubot wrote: post #12

Earlier quoted context omitted.

Not sure how "layman" to get... but here you go :) ASAN helps find difficult-to-reproduce bugs, e.g. data races. A data race occurs when two or more threads access the same memory locations in a undefined order. The OS can interrupt threads at any time, it's hard to test all possible interleavings of instructions from multiple threads. Programmers typically have some invariants in their mind but it's really easy to m…

What you described is Thread Sanitizer (clang toolchain). The corresponding valgrind-based tools would be Helgrind and DRD. ASan can only catch memory errors in C/C++ programs. A memory error is an access outside the allocated memory of an object, e.g. due to incorrect pointer arithmetics. Such errors would otherwise go undetected but can cause all kinds of errors in the program, like corrupted data, crashes etc.

There's some overlap between the two tools. Race conditions often lead to memory errors of the type that ASan can detect. TSan/Helgrind are the way to go if you want to find races that happen to be benign most of the time, or lock ordering problems, or races that lead to non-memory problems.
Post reply on HN