You could take a look at the proceedings of conferences like ICSE (International Conference on Software Engineering). For example there's been a lot of work over the last decade or so on making dynamic symbolic execution techniques more practical (as exemplified by e.g. KLEE or SAGE).
Ask HN: What is the emerging state of the art in fuzzing techniques?
11–20 of 54 posts
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#12I am very interested in learning about this too! I started out with AFL for fuzzing but soon had to move to LLVM's LibFuzzer because I didn't want non-ASCII inputs (by design, we know we wouldn't get that) and also SantizerCoverage seemed to be more robust than the 64kB shared memory array that AFL uses for large programs. However, libFuzzer being an in-process fuzzer has again created a lot of headache - especially…
Bear in mind that while AFL may feed you a binary string, you can "decode" that into whatever you want. It may take a bit of creativity or bit-shifting, but it can be done. You're free to grab 8 bits off the front and start setting global run-time flags or something, for instance. It's just data.
Also due to the way that AFL works, if you have a branch at the beginning of your program that immediately exits if it sees non-ASCII input, you don't lose all that much time, because AFL sees that as the same branch being exercised over and over again and hammers only on the input that allowed it to progress past that. In fact I think I almost always use AFL on text-based protocols, and it works fine. It's a common use case for AFL.
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#13I am very interested in learning about this too! I started out with AFL for fuzzing but soon had to move to LLVM's LibFuzzer because I didn't want non-ASCII inputs (by design, we know we wouldn't get that) and also SantizerCoverage seemed to be more robust than the 64kB shared memory array that AFL uses for large programs. However, libFuzzer being an in-process fuzzer has again created a lot of headache - especially…
Bear in mind that while AFL may feed you a binary string, you can "decode" that into whatever you want. It may take a bit of creativity or bit-shifting, but it can be done. You're free to grab 8 bits off the front and start setting global run-time flags or something, for instance. It's just data. Also due to the way that AFL works, if you have a branch at the beginning of your program that immediately exits if it see…
While I didn't internally decode the bitstring, you are right. AFL did generate tonnes of useful tests and I did uncover a couple of bugs using it.
That said, given that I didn't spend too much time actually trying to understand AFL, could you clarify if I was right in my understanding that AFL doesn't have true coverage but rather a heuristic using a table as documented in http://lcamtuf.coredump.cx/afl/technical_details.txt . Given that the program I was fuzzing was HUGE, wouldn't it falsely alias branches?
Thank you for your input!
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#14Here you have some interesting work from Fabien Duchene, ENSIMAG/CEA researcher, about black-box genetic fuzzing (I know it sounds like a lot of buzzwords, and in fact it was a little bit mocked during SSTIC 2016, but it's some really good stuff !) http://hal.univ-grenoble-alpes.fr/hal-00978844/ https://dl.acm.org.sci-hub.cc/citation.cfm?id=2557550&dl=ACM...
I don't know if adding a sci-hub link is a good idea.
Why not?
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#15I would have a look at Project Zero. P.S. I think many governments and corporations would keep their fuzzing techniques quite secret. You don't want to do the same fuzzing as anyone else.
Really? I thought everyone dropped "security by obscurity" long time ago.
contrary to popular opinion....
Obscurity is effective as one layer in a layered defense.
"Defense in depth".
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#16Earlier quoted context omitted.
Bear in mind that while AFL may feed you a binary string, you can "decode" that into whatever you want. It may take a bit of creativity or bit-shifting, but it can be done. You're free to grab 8 bits off the front and start setting global run-time flags or something, for instance. It's just data. Also due to the way that AFL works, if you have a branch at the beginning of your program that immediately exits if it see…
While I didn't internally decode the bitstring, you are right. AFL did generate tonnes of useful tests and I did uncover a couple of bugs using it. That said, given that I didn't spend too much time actually trying to understand AFL, could you clarify if I was right in my understanding that AFL doesn't have true coverage but rather a heuristic using a table as documented in http://lcamtuf.coredump.cx/afl/technical_de…
It could probably go either way, depending on the nature of your program. It depends on the correlation between "separate paths of execution" and "separate test cases". Certainly that's strong for a lot of programs, but if I sat down to construct pathological cases I probably could, and in the world of Turing chaos that programs inhabit, if pathological cases can exist, you can count on hitting them sometimes, no matter what low probability of that outcome you think you can justify.
AFL is definitely heuristic, and thus can conceivably be fooled in places where a true symbolic execution wouldn't be. On the other hand, it's very fast and easy to set up and use. Can't ever have it all. :)
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#17Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#18DARPA's Cyber Grand Challenge is pushing the state of the art when it comes to RE, exploitation, and the like. Very presentable talk by Mike Walker, who's heading up the project with a bunch of ex-Raytheon guys.
There are links to some repos of in the talk: it's not exactly what you're looking for, but if you're interested, it's a good resource.
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#19My friend is working on some next gen fuzzing stuff here:
Re: Ask HN: What is the emerging state of the art in fuzzing techniques?
#20curious how people who have been doing fuzz testing feel about property-based testing (at least the part about generating inputs, not asserting results)