Looks like a good start. Things can get complicated when going into request bodies, especially oneOf, anyOf relations.
I went through this journey while writing a similar tool: https://github.com/Endava/cats. Currently it has 72 Fuzzers registered which cover quite a lot of scenarios.
Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
11–20 of 25 posts
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#12There's another one here by Microsoft - this is cool though! great to see more Rust tools.
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#13Does it support OAS 3.1?
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#14Does it support OAS 3.1?
[deleted]
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#15Does it support OAS 3.1?
No, it does not. As for the parsing of the OAS, it relies on openapiv3[0], which does not yet support it.
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#16Great job! I wonder if that could be used with other fuzzers in OSS fuzz (giving tools more options to use for fuzzers and APIs)
Right now, it's probably not well suited for use in CI, since it runs until it's terminated. But I definitely plan to add a support for better usage in CI along with some examples. Thanks for the suggestion. I'll look into it!
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#17This is fantastic! I was a just looking for a fuzzer for OpenAPI stuff. I'm building a project with FastAPI that I'd love to try this on.
You can already do this using Hypothesis with the Pydantic plugin, https://pydantic-docs.helpmanual.io/hypothesis_plugin/. Here is an example for Quart-Schema (similar setup to FastAPI) https://pgjones.dev/blog/automatic-api-testing-2021.
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#18Does it support OAS 3.1?
No, it does not. As for the parsing of the OAS, it relies on openapiv3[0], which does not yet support it. 0: https://github.com/glademiller/openapiv3
That's too bad. It's starting to feel like 3.1 is never going to gain adoption unless we get entirely new tools built for it.
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#19Hi HN, I'd like to share with you a fuzzer I've been working on. It is a black-box, smart, generation-based fuzzer, that fuzzes APIs based on OpenAPI specification. It all started as a bachelor thesis[0], when I wanted to do something security-related and learn Rust along the way. My colleague @viralpoetry tutored me and so far, we've been able to find bugs in software such as k8s, gitea, and vault[1]. As for the cho…
Love this. Sad it’s GPL though.
Fun fact but there are very little hazards for GPL with testing software. Since it doesn't ship with the actual software it can be restricted to the source being distributed to the "users" of the tests which are the developers. If source must be provided to the contractor (in the case of say the US gov or a corporation), the tests only need be run in the view of a restricted set of individuals which allows them to receive the source but none of the standard users of the main, non-testing software.
Re: Show HN: OpenAPI fuzzer – fuzzing APIs based on OpenAPI specification
#20There's another one here by Microsoft - this is cool though! great to see more Rust tools. https://github.com/microsoft/restler-fuzzer
Yes, we looked into it. There is a chapter in my thesis[0] about RESTler and comparison with OpenAPI fuzzer. The main difference between those two fuzzer is that RESTler is a statefull fuzzer and OpenAPI fuzzer is a stateless fuzzer.
Thanks to being statefull, RESTler is able to analyze a dependencies between a requests. For example, it will not call and endpoint to get user details before calling endpoint to create a user. This should make it more efficient because it does not waste requests to calling endpoint that will return 404.
On the other hand, fuzzing is all about trying to supply unexpected input to the software. Therefore, OpenAPI fuzzer makes those requests, since it may cause some undefined behavior when we try to get user before creating one.
So while RESTler tries to check some invariant (for example if deleted user cannot be accessed) OpenAPI fuzzer tries to cause the service to crash by invalid input. RESTler is usually not able to crash the service by providing invalid input, because it needs to keep the fuzzing dictionary small (bigger dictionary would make the dependency finding slow). So I think, they complements each other nicely.
Another difference it in reporting error. RESTler considers only status code 500 as an error. However, OpenAPI specification states all possible status codes that we can get from an endpoint. OpenAPI fuzzer utilizes this and reports every time it receives and unexpected status code.
0: https://github.com/matusf/bachelor-thesis/releases/download/...