FFmpeg and a thousand fixes • Hacker News Board

Re: FFmpeg and a thousand fixes

#131

Post by yetanotherphd » Sat, Jan 11, 2014, 7:36 AM UTC

haberman wrote: post #66

pjmlp wrote: post #22

The wonders of C: - NULL pointer dereferences, - Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access, - Out-of-bounds reads and writes to stack, heap and static-based arrays, - Invalid free() calls, - Double free() calls over the same pointer, - Division errors, - Assertion failures, - Use of uninitialized memory. But hey, any good programmer always writes perfect C code.

Yeah, and you know, the Rust guys are actually doing some great work on making a language that solves some of these problems without giving up the advantages that make C programmers choose C. Meanwhile, you are just being snarky on a message board while using a whole stack of software that was built by the programmers whose work you are criticizing.

I don't think the poster was being snarkey, they were merely pointing out how hard it is to write correct C code. And there was no criticism of the developers, in fact the post implied that the bugs were the fault of C and not the developers.

Re: FFmpeg and a thousand fixes

#132

Post by CraigJPerry » Sat, Jan 11, 2014, 8:39 AM UTC

So my paranoia has finally taken over. My first read through and I only take in these features:

    * I presume all this infra is used for more than just ffmpeg so why this software specifically? Im conscious its included in everthing from web browsers to games. If this was an attack vector, its a BIG vector.
    * 1121 (or whatever it was) is a curious number to make a post about. Why not at the 1k mark or 1250?
    * Explicitly calling out use of an unpriv user to a readership likely already well versed in basic security practices but notorious for casually ignoring on personal machines

Google, blink twice if theyve got ya tied up in some legal requirement that you cant disclose.

Re: FFmpeg and a thousand fixes

#133

Post by pjmlp » Sat, Jan 11, 2014, 9:09 AM UTC

thristian wrote: post #114

lhgaghl wrote: post #103

Earlier quoted context omitted.

If Java is somehow a problem, you can simply make a language without stack smashing but also without GC. Nobody has done this so far because nobody gives a shit about the fact that every program we use is full of remote code vulns and will never be fixed.

You haven't been reading all the Rust release announcement threads, have you?

The thing is, those languages do exist since C does.

But the OSs created with them weren't as fancy as UNIX was at the time, or used hardware more costly than the PDP-11.

Re: FFmpeg and a thousand fixes

#134

Post by pjmlp » Sat, Jan 11, 2014, 9:17 AM UTC

rossy wrote: post #111

pjmlp wrote: post #22

The wonders of C: - NULL pointer dereferences, - Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access, - Out-of-bounds reads and writes to stack, heap and static-based arrays, - Invalid free() calls, - Double free() calls over the same pointer, - Division errors, - Assertion failures, - Use of uninitialized memory. But hey, any good programmer always writes perfect C code.

And yet, FFmpeg couldn't really have been written in anything else. A 5% difference in performance could be the difference between a 1080p video decoding in real time on a low powered device like a phone or a netbook and the video decoding slightly slower than real time. Improvements to FFmpeg made my old netbook work as a video player. I don't think I would have been able to use an FFmpeg-based player at all if it h…

Those 5% are written in Assembly anyway. C does not expose SIMD, non portable compiler specific extensions do.

For the remaining 95% of the code, since 1978 one could use Modula-2, Ada, Pascal dialects, SafeC and many other system programming languages.

Back in the early 80's, C compilers were as crap as any other system programming languages when generating code for home computers.

Re: FFmpeg and a thousand fixes

#135

Post by userbinator » Sat, Jan 11, 2014, 12:25 PM UTC

pjmlp wrote: post #22

The wonders of C: - NULL pointer dereferences, - Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access, - Out-of-bounds reads and writes to stack, heap and static-based arrays, - Invalid free() calls, - Double free() calls over the same pointer, - Division errors, - Assertion failures, - Use of uninitialized memory. But hey, any good programmer always writes perfect C code.

From one of the linked blogs containing details:

> There were lots of dynamic allocations

This surprises me; a lot of codecs are specified in such as way as to not require dynamic allocation at all.

I hypothesise that if it was written in another language, we would just see bugs appear in a different form.

Re: FFmpeg and a thousand fixes

#136

Post by panzi » Sat, Jan 11, 2014, 1:47 PM UTC

nitrogen wrote: post #130

astrange wrote: post #129

Earlier quoted context omitted.

Nearly all SIMD in ffmpeg/libav/x264 is just written in assembly. Back in the day, gcc's code generation for even x86 SIMD intrinsic functions was too poor to consider using it. (Plus x86's intrinsics are just plain ugly. It's easier to read asm.) As for the architecture-independent SIMD in GNU C, it's something, but not quite flexible enough. Even C is really not a good enough language to consider implementing this…

Isn't that what the restrict keyword is for?

Yes, but how long does this exist? My guess would be that it isn't really used (yet) in ffmpeg.

Re: FFmpeg and a thousand fixes

#137

Post by panzi » Sat, Jan 11, 2014, 1:50 PM UTC

colanderman wrote: post #65

pjmlp wrote: post #51

Earlier quoted context omitted.

That is a compiler issue, not language. ISO/ANSI C also don't support SIMD, you have to go down to Assembly when writing portable code across C compilers.

Ya, it's definitely a language issue; barring super-intelligent compilers that can vectorize things on their own (which GCC does for certain cases), you need a means to express vectorizable operations. You don't need to work at the assembly level for this; you just need a language that can express vector operations in a way that doesn't require a compiler to solve the halting problem, and you need a compiler that kno…

Well, expressions like these look to me very clearly like a pattern that can be implemented using SIMD instructions:

    map (uncurry (+)) (zip [1,2,3] [4,5,6])

I guess because of these standard functions and the fact that everything in Haskell is immutable and side effect free it should be comparably easy to do SIMD optimizations. It's only a guess, though.

Re: FFmpeg and a thousand fixes

#138

Post by neves » Sat, Jan 11, 2014, 4:10 PM UTC

The software testing course in Udacity is an excellent introduction to the fuzzy testing techniques that the article talks about. Here is the link: https://www.udacity.com/course/cs258

Re: FFmpeg and a thousand fixes

#139

Post by npsimons » Sat, Jan 11, 2014, 4:51 PM UTC

scott_s wrote: post #95

gcr wrote: post #94

Earlier quoted context omitted.

This list of criticism is copied verbatim from the article's list of fixed bugs, not composed directly by the OP.

The snark, however, is wholly original.

Considering it's entirely predictable and I've seen ones just like it a million times before, no, it's not. Relevant xkcd, esp. panel 2: https://xkcd.com/568/

"You will never find a programming language that frees you from the burden of clarifying your ideas."

Re: FFmpeg and a thousand fixes

#140

Post by npsimons » Sat, Jan 11, 2014, 4:59 PM UTC

userbinator wrote: post #135

pjmlp wrote: post #22

The wonders of C: - NULL pointer dereferences, - Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access, - Out-of-bounds reads and writes to stack, heap and static-based arrays, - Invalid free() calls, - Double free() calls over the same pointer, - Division errors, - Assertion failures, - Use of uninitialized memory. But hey, any good programmer always writes perfect C code.

From one of the linked blogs containing details: > There were lots of dynamic allocations This surprises me; a lot of codecs are specified in such as way as to not require dynamic allocation at all. I hypothesise that if it was written in another language, we would just see bugs appear in a different form.

I hypothesise that if it was written in another language, we would just see bugs appear in a different form.

Exactly! Programming is hard to get right, no matter what the language. I hate these people who come along saying "hurr, durr, C sucks, project X should have been written in language Y!" All the while ignoring the facts that a) they can go and rewrite X in Y any time they like, and b) if it was written in Y, people would be bitching about issues with Y (eg, speed). Shortsightedness and lack of will to put your code where your mouth is are two of the most despicable traits a programmer can have.