Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg and a thousand fixes

googleonlinesecurity.blogspot.com

Post reply on HN
Print view View original

121–130 of 150 posts

Re: FFmpeg and a thousand fixes

#121

Post by lhgaghl »

pcwalton wrote: post #109
ajross wrote: post #108

Earlier quoted context omitted.

My (micro-) point was more that memory bugs are going to be inherent in a codec implementation because of the nature of hte problem, obviously not that all of the bugs are going to come from the assembly. But the macro point is just empirical: the proof that Rust is an appropriate tool for codec development needs to come in the form of a production-quality codec implementation of some non-trivial standard (VP9, say),…

> My (micro-) point was more that memory bugs are going to be inherent in a codec implementation because of the nature of hte problem, obviously not that all of the bugs are going to come from the assembly. I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem. > But the macro point is just empirical: the proof that Rust is an appropriate to…

> For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler.

...or just add more primitives to the language.

Re: FFmpeg and a thousand fixes

#122

Post by tttt1111 »

samworm wrote: post #2

It is interesting that YouTube isn't mentioned in this blogpost, despite there being good evidence that ffmpeg has been used there[1]. The fuzz testing they mention is based around constructing malformed (or at least "exotic") input files and then monitoring for failures... ie simulating exactly the kind of attack someone might use against YouTube's transcoding infrastructure. [1] http://multimedia.cx/eggs/googles-yo…

Google has never officially acknowledged that YouTube uses ffmpeg. I seem to recall it was because of legal reasons they are not allowed to. So you will never hear a Google employee mention anything about YouTube and ffmpeg ;)

Re: FFmpeg and a thousand fixes

#123

Post by dbaupp »

hrjet wrote: post #118
pcwalton wrote: post #109

Earlier quoted context omitted.

> My (micro-) point was more that memory bugs are going to be inherent in a codec implementation because of the nature of hte problem, obviously not that all of the bugs are going to come from the assembly. I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem. > But the macro point is just empirical: the proof that Rust is an appropriate to…

> I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem. For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler.

Rust allows this via `unsafe` blocks, which essentially have C semantics (and allow the use of inline assembly too).

Re: FFmpeg and a thousand fixes

#124

Post by dbaupp »

lhgaghl wrote: post #121
pcwalton wrote: post #109

Earlier quoted context omitted.

> My (micro-) point was more that memory bugs are going to be inherent in a codec implementation because of the nature of hte problem, obviously not that all of the bugs are going to come from the assembly. I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem. > But the macro point is just empirical: the proof that Rust is an appropriate to…

> For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler. ...or just add more primitives to the language.

Why make the language larger when you can write identical things in libraries? (For Rust, you can use `unsafe` blocks to write very low-level primitives in normal user code and then expose a safe interface above them.)

Re: FFmpeg and a thousand fixes

#125

Post by [deleted] »

dbaupp wrote: post #123
hrjet wrote: post #118

Earlier quoted context omitted.

> I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem. For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler.

Rust allows this via `unsafe` blocks, which essentially have C semantics (and allow the use of inline assembly too).

[deleted]

Re: FFmpeg and a thousand fixes

#126

Post by fzltrp »

dbaupp wrote: post #124
lhgaghl wrote: post #121

Earlier quoted context omitted.

> For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler. ...or just add more primitives to the language.

Why make the language larger when you can write identical things in libraries? (For Rust, you can use `unsafe` blocks to write very low-level primitives in normal user code and then expose a safe interface above them.)

It depends on what you guys are talking about.

If you mean "constant" primitives, i.e. things which don't require to expand the language grammar or type system, then it's a question of future prevalence of usage of said primitives. If they become almost ubiquitous to any project, then they obviously need to be included in the standard libs. If you're talking about a new language feature, that depends on how much that new feature may help design and write future libraries, and also what are the aims of said language. I understand that rust aims to become a safer system language, so any feature helping the design of low level, performance critical code should be welcome.

Re: FFmpeg and a thousand fixes

#127

Post by dpe82 »

tttt1111 wrote: post #122
samworm wrote: post #2

It is interesting that YouTube isn't mentioned in this blogpost, despite there being good evidence that ffmpeg has been used there[1]. The fuzz testing they mention is based around constructing malformed (or at least "exotic") input files and then monitoring for failures... ie simulating exactly the kind of attack someone might use against YouTube's transcoding infrastructure. [1] http://multimedia.cx/eggs/googles-yo…

Google has never officially acknowledged that YouTube uses ffmpeg. I seem to recall it was because of legal reasons they are not allowed to. So you will never hear a Google employee mention anything about YouTube and ffmpeg ;)

We do at least know that they use x264: http://x264licensing.com/adopters

Re: FFmpeg and a thousand fixes

#128

Post by fzltrp »

al2o3cr wrote: post #104
Perseids wrote: post #48

Earlier quoted context omitted.

On the 30C3 there was a talk about a C Compiler (actually an llvm optimization plugin) that can eliminate nearly every memory management related vulnerability by adding memory checks. The penalty is only a 100% increase in runtime. > If you're a C programmer and somebody says like: I've that optimization that makes your program 3% faster you go: wow! Then you come and say: well, now I make it half as fast you go: w00…

"that can eliminate nearly every memory management related vulnerability" This is like saying "my bulletproof vest will stop nearly every round fired at it". Too bad it only takes ONE...

So I suppose you'd rather not wear any bulletproof vest?

Re: FFmpeg and a thousand fixes

#129

Post by astrange »

colanderman wrote: post #65
pjmlp wrote: post #51

Earlier quoted context omitted.

That is a compiler issue, not language. ISO/ANSI C also don't support SIMD, you have to go down to Assembly when writing portable code across C compilers.

Ya, it's definitely a language issue; barring super-intelligent compilers that can vectorize things on their own (which GCC does for certain cases), you need a means to express vectorizable operations. You don't need to work at the assembly level for this; you just need a language that can express vector operations in a way that doesn't require a compiler to solve the halting problem, and you need a compiler that kno…

Nearly all SIMD in ffmpeg/libav/x264 is just written in assembly. Back in the day, gcc's code generation for even x86 SIMD intrinsic functions was too poor to consider using it.

(Plus x86's intrinsics are just plain ugly. It's easier to read asm.)

As for the architecture-independent SIMD in GNU C, it's something, but not quite flexible enough. Even C is really not a good enough language to consider implementing this in, because it has an overly-conservative memory model when it comes to char *, which aliases all other memory. For a library dedicated to parsing things which come in the form of bytestreams and 8-bit pixels, this means that most compiler optimizations are defeated right in the inner loops where you'd need them.

Re: FFmpeg and a thousand fixes

#130

Post by nitrogen »

astrange wrote: post #129
colanderman wrote: post #65

Earlier quoted context omitted.

Ya, it's definitely a language issue; barring super-intelligent compilers that can vectorize things on their own (which GCC does for certain cases), you need a means to express vectorizable operations. You don't need to work at the assembly level for this; you just need a language that can express vector operations in a way that doesn't require a compiler to solve the halting problem, and you need a compiler that kno…

Nearly all SIMD in ffmpeg/libav/x264 is just written in assembly. Back in the day, gcc's code generation for even x86 SIMD intrinsic functions was too poor to consider using it. (Plus x86's intrinsics are just plain ugly. It's easier to read asm.) As for the architecture-independent SIMD in GNU C, it's something, but not quite flexible enough. Even C is really not a good enough language to consider implementing this…

Isn't that what the restrict keyword is for?
Post reply on HN