Two months and 1100+ commits to rediscover a bug that was already found in 2024 is probably the funniest possible ending to a "vibecoded fuzzer" story.
[deleted]
We found a division by zero bug in FFmpeg with a vibecoded fuzzer
271–274 of 274 posts
Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer
#272IDK seems like a bug that could've taken a human a few minutes at best to find. I found a bug in SystemD that would crash the daemon because a bad SystemD unit file configuration. That took me like 5 minutes to actually track down in the actual source code. I understand the utility of this though, I just don't see this particular bug and something that would be particularly difficult o find pre LLM era.
This is the wrong mentality. The fuzzer found the bug before any humans did, so there is a mismatch of developers who could find this bug and those who did (without an LLM-coded fuzzer). The value of the fuzzer continues long after it found this one bug. It's worth nothing that in the bug discussion thread, the bug fix author pointed out that it's not easy to set up the config then call the functions in the right ord…
But I am being very specific to this use case, where a division by zero bug was found. Why couldn't you have just grepped through the codebase, found all possible divisions and ensured that they had a check on it to never be less than or equal to 0?
The bug I located in SystemD was literally a null ptr exception. All they had to do was perform a null check on a cstring but they hadn't.
I don't see the utility of reporting an LLM made fuzzer finding bugs that could be found by a lint rule or static analysis. I will appreciate a post about an LLM fuzzing software to find unique corner cases, which I predict will happen soon, in ACL controlled systems caused by policy shadowing.
Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer
#273IDK seems like a bug that could've taken a human a few minutes at best to find. I found a bug in SystemD that would crash the daemon because a bad SystemD unit file configuration. That took me like 5 minutes to actually track down in the actual source code. I understand the utility of this though, I just don't see this particular bug and something that would be particularly difficult o find pre LLM era.
Bugs are always easier to find in retrospect.
This is literally the lowest rung of a bug, it's equivalent to finding a nullptr exception. You can literally avoid them with 1 if statement.