Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Day I Fell in Love with Fuzzing

nullprogram.com

Post reply on HN
Print view View original

11–20 of 49 posts

Re: The Day I Fell in Love with Fuzzing

#11

Post by adrianN »

ergothus wrote: post #6

> When I got started, I had just learned how to use yacc (really Bison) and lex (really flex) I've dug into parsers a few times, but everything I encountered seemed to think that once I had a parsed tree of commands it was obvious how to consume it...and it wasn't (for me). I've never had the free time to dedicate to experimenting that abstractly, so anytime I'm tempted to write a DSL or similar for a current problem…

IMHO you're often best off with writing a simple recursive descent parser by hand. For lexing you can often get away with splitting the input string at word boundaries.

For slightly more advanced needs, parser combinator libraries make parsing and lexing quite straightforward. I honestly wouldn't use a parser generator.

Re: The Day I Fell in Love with Fuzzing

#12

Post by detaro »

Quarrelsome wrote: post #10
c256 wrote: post #5

Earlier quoted context omitted.

Yes, because retro-gaming is a thing.

Right. I'm just saying why not rip it out now? Why bother fuzzing parsers if the parsers themselves aren't really adding value anymore? The ability to edit your configs in a text editor is pretty valuable isn't it? Maybe he's just got used to it.

He isn't rewriting the game (that was made by someone else), he is rewriting his tools for modifying the game files.

Re: The Day I Fell in Love with Fuzzing

#13

Post by amenod »

ergothus wrote: post #6

> When I got started, I had just learned how to use yacc (really Bison) and lex (really flex) I've dug into parsers a few times, but everything I encountered seemed to think that once I had a parsed tree of commands it was obvious how to consume it...and it wasn't (for me). I've never had the free time to dedicate to experimenting that abstractly, so anytime I'm tempted to write a DSL or similar for a current problem…

Actually "compilers course" is exactly what you should be looking for imho. Compilers are basically apps that parse input and output something in response to it - which is what this is all about. Lex and yacc are very simple to use once you understand the core principles.

Re: The Day I Fell in Love with Fuzzing

#15

Post by afraca »

The test files are coupled tightly to the implementation. He says it himself that when he wants to restructure things new tests have to be generated. It seems clumsy, but I don't have strong negative feelings on it.

Re: The Day I Fell in Love with Fuzzing

#16

Post by lmm »

ergothus wrote: post #6

> When I got started, I had just learned how to use yacc (really Bison) and lex (really flex) I've dug into parsers a few times, but everything I encountered seemed to think that once I had a parsed tree of commands it was obvious how to consume it...and it wasn't (for me). I've never had the free time to dedicate to experimenting that abstractly, so anytime I'm tempted to write a DSL or similar for a current problem…

I could never understand why you have a parser as a separate thing when this was presented as an up-front decision. Reading this series of posts, where something like a traditional compiler architecture emerges in the same way that normal program architecture emerges via gradual refactoring, made it make a lot more sense to me: https://hokstad.com/compiler .

Re: The Day I Fell in Love with Fuzzing

#17

Post by kragen »

I haven't tried afl-fuzz myself, although it sounds like world-class awesome software, but I'm a real believer in testing things with David MacIver's Hypothesis, which invokes your functions with random inputs, and then does similar canonicalization and minimization kinds of things. I like Hypothesis so much that when I wrote Dumpulse http://github.com/kragen/dumpulse I added a Python interface to it purely so I could test it with Hypothesis. Which found bugs, of course, even though Dumpulse is around 100 instructions when compiled.

Re: The Day I Fell in Love with Fuzzing

#18

Post by rictic »

ergothus wrote: post #9
jerf wrote: post #8

Earlier quoted context omitted.

What skills/languages do you have? It can help tune the suggestions. Barring that, you can google "compiler tutorial" and just start looking for one you like. One nice thing about compilers is they consist of a lot of stages and while the whole is arguably greater than the sum of its parts, the parts are all pretty darned useful on their own, too.

Javascript is my strongest, and I have enough Python to be comfortable - I had a lot of Perl back in the day but that knowledge is fairly rusty (though Parse::RecDescent was my first exposure to parsers). I did Java for a few years, but I refer to those as my Dark Times and wouldn't want to venture back there. (Experienced Perl dev doing Java is NOT a good time for either the dev or the code) I can't argue about the…

If you got to the point of having a tree, but found them clumsy to work with, the first thing I'd suggest is to play with an AST explorer. https://astexplorer.net/ is quite good.

Picture the task of what you want to do in your head. Maybe you're writing a linter, and you want to enforce a rule like "never use the 'var' keyword". Write some example code and poke around in the explorer.

Another thing to keep in mind is that once you have a tree, usually you also want to have some tree traversal utilities, so that you can walk over the tree and optionally transform it.

Re: The Day I Fell in Love with Fuzzing

#20

Post by sametmax »

kragen wrote: post #17

I haven't tried afl-fuzz myself, although it sounds like world-class awesome software, but I'm a real believer in testing things with David MacIver's Hypothesis, which invokes your functions with random inputs, and then does similar canonicalization and minimization kinds of things. I like Hypothesis so much that when I wrote Dumpulse http://github.com/kragen/dumpulse I added a Python interface to it purely so I coul…

I love the concept of hypothesis, but I struggle with finding a real life use case. I just can't formulate my assertions, I don't know what to put in them.

unit tests are easy since I know what the code does, and I can just tell it what to do and what I expect.

But with hypothesis, I have to find some kind of general property, which I have a hard time to do.

Any tips, or materials I can use ?

P.S: I read your test.py, and see you use a rule base state machine. I've never seen that in any tutorial on hypothesis I read. What does it do ? How do you use it ?

Post reply on HN