Live data from Hacker News

Ask HN: What is the emerging state of the art in fuzzing techniques?

news.ycombinator.com

1–10 of 54 posts

Ask HN: What is the emerging state of the art in fuzzing techniques?

#1
I'm fairly familiar with the popular tools such as afl and Codenomicon Defensics. But I find the academic literature very opaque and don't really know where to start.

If I want to understand the cutting edge of fuzzing techniques, and what will be the emerging state of the art in the next few years - where should I look? Any good papers or books (with at least some for novices to understand), or research projects that are leading towards a new excellence?

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#3
I am very interested in learning about this too! I started out with AFL for fuzzing but soon had to move to LLVM's LibFuzzer because I didn't want non-ASCII inputs (by design, we know we wouldn't get that) and also SantizerCoverage seemed to be more robust than the 64kB shared memory array that AFL uses for large programs.

However, libFuzzer being an in-process fuzzer has again created a lot of headache - especially in places where we malloc stuff and expect free to implicitly happen at exit - in libFuzzer's case, the exit is caught and the entrypoint function is restarted causing memory leaks and OOM crashes. This made me have to include #ifdef FUZZ ... #endif lines in the codebase - adding different behavior in fuzzed and unfuzzed cases which felt wrong.

I have considered implementing an out-of-process fuzzer from scratch (or base it off AFL), but have been holding off till I get time to read about more prior work given that this is not of the highest priority at work.

That said, SAGE from Microsoft seems really interesting[1]. It generates inputs intelligently by constraint-solving on inputs to conditional statements. It isn't exactly new though.

[1] http://research.microsoft.com/en-us/um/people/pg/public_psfi...

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#5
post #4

I would have a look at Project Zero. P.S. I think many governments and corporations would keep their fuzzing techniques quite secret. You don't want to do the same fuzzing as anyone else.

Really?

I thought everyone dropped "security by obscurity" long time ago.

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#6
This is not exactly what you asked for, but if the code you want to fuzz is written by yourself, you could learn fuzzing by doing the fuzzing yourself. It might also be easier to do this at first, since you're closer to your code, and would need less adapters for existing fuzz solutions.

1. Write a simple test function which will generate a very wide range of allowed inputs to the function you want to test. Try to generate average inputs most of the time, and outliers some of the time. Use a seeded Mersenne Twister as your random number generator. For example, if the function you are testing accepts an array of buffers, then for a single test of the function, you could choose at random how many buffers to generate, and then at random the length of each buffer, and then at random the contents of each buffer. You could then call the function many times, each time with a different array of inputs. Or if you were testing a document editor or CRDT, you might want to randomly generate different combinations of user edits, e.g. a delete 10% of the time, an insert 50% of the time, etc.

2. Write the simplest possible independent implementation of the function you want to test. For example, if you are testing a custom hash map, you could use the hash map from your standard library as the basis for the independent implementation. Or if you were testing a key/value storage engine, you could consider using an in-memory hash as the basis for the independent implementation.

3. Run your random fuzz inputs from step 1 through both your implementations and assert that the outputs of both are always the same at each step. Both implementations could be called a few thousand times depending on the run time.

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#7
post #5
post #4

I would have a look at Project Zero. P.S. I think many governments and corporations would keep their fuzzing techniques quite secret. You don't want to do the same fuzzing as anyone else.

Really? I thought everyone dropped "security by obscurity" long time ago.

By itself, sure: but it can only help if used in tandem with other methods

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#8
Here you have some interesting work from Fabien Duchene, ENSIMAG/CEA researcher, about black-box genetic fuzzing (I know it sounds like a lot of buzzwords, and in fact it was a little bit mocked during SSTIC 2016, but it's some really good stuff !)

http://hal.univ-grenoble-alpes.fr/hal-00978844/ https://dl.acm.org.sci-hub.cc/citation.cfm?id=2557550&dl=ACM...

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#9

Here you have some interesting work from Fabien Duchene, ENSIMAG/CEA researcher, about black-box genetic fuzzing (I know it sounds like a lot of buzzwords, and in fact it was a little bit mocked during SSTIC 2016, but it's some really good stuff !) http://hal.univ-grenoble-alpes.fr/hal-00978844/ https://dl.acm.org.sci-hub.cc/citation.cfm?id=2557550&dl=ACM...

I don't know if adding a sci-hub link is a good idea.

Re: Ask HN: What is the emerging state of the art in fuzzing techniques?

#10
post #5
post #4

I would have a look at Project Zero. P.S. I think many governments and corporations would keep their fuzzing techniques quite secret. You don't want to do the same fuzzing as anyone else.

Really? I thought everyone dropped "security by obscurity" long time ago.

I was actually thinking along the lines of fuzzing for exploit development.

You want a unique bug that will last a long time. In which case, your fuzzing techniques are a trade secret. A lot of fuzzing advances take place behind closed doors.

Project Zero has some people with interesting backgrounds doing bug hunting for good.

Post reply on HN