Live data from Hacker News

We found a division by zero bug in FFmpeg with a vibecoded fuzzer

code.ffmpeg.org

61–70 of 274 posts

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#61
post #7

Earlier quoted context omitted.

You're not reading it right. The bug was found using a vibecoded fuzzer.

I wonder from where Claude stole this fuzzer.

The very first line of the bug report:

> This is a bug found with our fuzzer: https://github.com/daedalus/fuzzer/

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#62

What we need is a numeric type that cannot be zero.

The only way to achieve this is to either put a runtime software check on a variable whenever it's assigned/used, or to literally add hardware support in processors themselves which literally throws an interrupt when a "neverShallBeZero" variable is assigned to zero.

There's no viable way to statically prove at compile-time that these variables will never become zero at runtime, ultimately forcing a system of endless runtime checks (be it software or hardware)... which is why processors already throw exception interrupts when division by zero is attempted.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#63
I imagine the discussion will center around this application of AI, but to me this is just the Nth proof of the proven fact that you must build ffmpeg, if you insist on using it, with only an allow-list of file formats that you expect to encounter, and not with the kitchen sink of stuff you are never going to need.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#64
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

What's stronger than Haskell?

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#65
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

Imo, formal methods like more expressive/stricter type systems are key to making LLM generated code successful. Of course models will get better, but trusting the output will become much easier with a type system that proves more properties.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#66

Earlier quoted context omitted.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

What's stronger than Haskell?

Perhaps coq/agda/idris/etc.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#67
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

Haskell's type system would not easily prevent this bug. It's not good at numeric/logic issues like that. When people say "Haskell makes it impossible to write bugs" they mean "Haskell has enums" (ADTs).

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#68

Earlier quoted context omitted.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

What's stronger than Haskell?

Anything with ranged numeric types. Like everyone's favorite functional programming language, Ada.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#69

Earlier quoted context omitted.

This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.

What's stronger than Haskell?

Dependent types is one possible direction. Not sure when a language with dependent types will arise which will be useful for making real programs.

Agda is the most mature dependently typed programming languae (having been around since the 90s – it is basically Haskell on steroids), but has a more proof-assistant flavor than an actual programming language flavor. Opus & Fable write Agda quite well, so LLMs can understand dependent types.

Re: We found a division by zero bug in FFmpeg with a vibecoded fuzzer

#70
post #4

Earlier quoted context omitted.

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

No one can keep up with the volume of code AI produces. We wont stop using AI. We will use AI to check AI. Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.

In fairness at root this has been going on for awhile. No one can keep up with the volume of machine code that modern more abstracted codebases produce.

We didn't stop using syntactic programming languages we used code to check code.

Not sure it's really crazy at all. It's been an abstraction for programmers probably since we stopped soldering transistors to each other.

Post reply on HN