21–30 of 32 posts
I wrote a blog post a while ago about the purpose of the lib, it's called "Completing the JavaScript Test Stack". Worth reading if you wonder what this is all about.
http://redotheweb.com/2014/01/07/completing-the-js-test-stac...
I think this is essentially "Fuzz testing"[1]. Something I read about in a book years ago but never saw done in real-life. The term may be antiquated at this point. [1] http://en.wikipedia.org/wiki/Fuzz_testing
I'm not sure how up to date the CVE list is, but it probably gives you an idea, if the fuzzing is still relevant or not :-)
Earlier quoted context omitted.
I wouldn't call this fuzz test because it focuses on simulating legitimate actions. Do random clicking. Do random scrolling. Do random mouseovers. Do random [any DOM event]. Yes, they are doing this at a massive scale, but the key here is that germlin.js can only do things a user can do. In short, Gremlin is just automating actions; basic legal, legitimate actions. Still cool, and great for stress testing or looking…
Not entirely true. Fuzzing is not about modifying data. I don't think that's the right way to describe fuzzing. It's about testing the durability of a program by trying all kinds of data, as greedy as possible. This includes known problematic inputs and random inputs to match some expectations. By random it can either be totally random (any length, any pattern) or protocol-aware. Random clicking is a form of random d…
Relating to the practical example: if one is trying to find bugs from the UI code (or it is the only way to feed inputs to the program), monkey method of fuzzing is the way to go. But if one tries to test the deeper layers of the program, it is beneficial to try to find the lowest layer of inputs we can access, since it enables faster input of test cases and thus makes the fuzzing more effective.
One way of the other, my opinion is that both are fuzzing, by the definition I gave for it :-)
That looks really cool, though I think it would have a hard time navigating past the first few menus, and could hit features that are only rarely meant to be touched (e.g. "delete everything".) Perhaps a searching algorithm that intentionally tries to explore every menu and feature, navigate to them and then test them. Or intentionally tries things it knows are more likely to find bugs.
Relevant doc: https://github.com/marmelab/gremlins.js#configuring-gremlins
For those who don't know: this is PROBABLY named after Palm OS Gremlins. Palm OS had a debugging tool called "gremlins" that would randomly tap and type and gesture all over your Palm OS app. This debugging tool would often find edge cases that would end up crashing your app. As someone who used to program for Palm OS a lot, I was pretty happy to see this name. :)
At first I was very skeptical that it would find any meaningful bugs, but in fact it was very good at finding memory leaks and edge cases. In the end it took us several weeks to finally be able to survive the 1M gremlins without a crash. A single run of 1M gremlins would take about 24 hours.
In later years they added the capability to supply your own dictionary of values to apply to certain inputs so that gremlins could get past validation, and it would also save the entire state of the device every 10K events so that you could easily reproduce errors that Gremlins found.
I could definitely see this as a good way to stress-test certain parts of your app, though it takes a ton of time, and frankly without surrounding tooling (like resource monitoring, saving "state") it wouldn't be practical.
Earlier quoted context omitted.
Not entirely true. Fuzzing is not about modifying data. I don't think that's the right way to describe fuzzing. It's about testing the durability of a program by trying all kinds of data, as greedy as possible. This includes known problematic inputs and random inputs to match some expectations. By random it can either be totally random (any length, any pattern) or protocol-aware. Random clicking is a form of random d…
One nice way of describing fuzzing is that it is running a search problem (searching for bugs) using more or less guided Monte Carlo method for the program's input space. Generally the testing should also be automated (test case injection and testing oracle), so that one can beat the program with enough test cases for the fuzzing to be viable. Relating to the practical example: if one is trying to find bugs from the…