Live data from Hacker News

We found a division by zero bug in FFmpeg with a vibecoded fuzzer

code.ffmpeg.org

31–40 of 274 posts

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#31
post #24

No doubt fuzzers (vibecoded or otherwise) can be powerful, but can't you just mark all "/" as potential divide by zero errors? I guess sometimes developers think they "know" some variable won't be zero, but unless it checked explicitly or by the compiler, that shouldn't be trusted.

Fuzzers find inputs, not just "potential" errors that aren't triggerable.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#33
post #4
post #2

It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

The missing part of this is that verifying the bug with LLMs is also easy, and so is adversarially reviewing the proposed fix with LLMs.

The only thing left for you to do should be directional decisions. The LLMs should pause and rope you in if the fix involves directional/invariant changes.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#34
post #4
post #2

It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

No one can keep up with the volume of code AI produces.

We wont stop using AI.

We will use AI to check AI.

Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#36
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

No one can keep up with the volume of code AI produces. We wont stop using AI. We will use AI to check AI. Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.

You're suggesting that LLMs get better at fixing bugs/vulnerabilities, but at the same time stop getting better at finding them? What if this difference is inherent and essential?

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#37

What we need is a numeric type that cannot be zero.

It would be more flexible for a compiler to reuse the range analysis logic used in optimizations for statically verifiable divide by zeros. That way you could extend it to other things like statically verifiable overflows.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#38
post #2

It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.

I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.

A.I. is useful for this. But it would be even more useful if all new code were written in Rust or some other memory-safe language.

A.I. could also be used to port C/C++ codebases to Rust, which isn't economically feasible at the moment.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#39
post #2

It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.

> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. No big deal? It’s not like it’s free… tokens cost money.

Often rounds to free compared to human costs.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#40
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

No one can keep up with the volume of code AI produces. We wont stop using AI. We will use AI to check AI. Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.

It's mostly (not entirely, but mostly) finding security issues in old human-written code. It'll eventually start running out of those.

From that standpoint, it's not a crazy setup security-wise. Maybe still crazy for development.

Post reply on HN