FFmpeg is 20 years old today

FFmpeg is a miraculous tool and I have relied on it for a couple decades now. Beyond the plethora of personal use-cases, it has enabled individuals and smaller companies to possess the same abilities as the bigger guys, who themselves are all pretty much using FFmpeg these days. The only folks who need to build/use alternatives are the platforms at the top of the market and/or with immense scaling needs who are truly pushing beyond the realm of FFmpeg's practical abilities. Twitch is one of those very few, and have been nicely open in discussing it. [1] [2]

Couple highlights for my careering using it over the years. The first was when I was working on production and post-production for a small studio that had a popular web series and was just about to transition to their first "big" shows that would be produced for Hulu. This was when 6K raw video was just becoming a thing, we had over 50tb of footage, GPU decoding was brand-new, Windows machines couldn't work practically with Apple ProRes, lots of challenges. I ended up building a system that did things like transcode raw footage into various formats automatically whenever the server noticed there was new footage, automatically collected and stored the metadata from every shot somewhere we could centrally browse/search/filter it, etc. When it came time to deliver, it would automatically create various outputs for the web. We had to deliver ProRes masters in the end and had recently transitioned entirely to PCs. This was around when somebody successfully implemented a pretty good ProRes encoder for FFmpeg, so we were then able to encode and deliver these huge ProRes outputs not only without needing a Mac, but also entirely on our servers, no longer requiring someone's workstation to be hijacked for an entire day to do this. It all may not seem too revolutionary, but there was no way we would've been able to work with the same efficiency, for the same cost, in the same timeframe otherwise.

A couple years later, at a new (now defunct) video platform with millions of videos and maybe 5 back-end engineers, FFmpeg allowed us to build our own service to encode all uploads into the many resolutions and formats required. Encoding services were (and still are) very expensive, but in just a couple weeks we had our own that ran on standard Ubuntu server instances, spinning up/down depending on load. Immense cost savings, and not tied to any particular company. Shortly thereafter, GPU instances were available from most cloud providers and `nvenc` was available in FFmpeg, so we were able to dramatically speed up the encode process with maybe a day of work by adding GPU encoding into the mix.

These may seem like pretty obvious possibilities now, but it cannot be overstated how insane it was, especially at the time, for tiny and/or cash-strapped teams to be able to do all of this so easily, and that at the tool at the crux of it all, FFmpeg, was completely free. Yes, FFmpeg can be a pain in the ass to figure out, and it's easy these days to take it for granted, but in my opinion it has been truly revolutionary.

[1] https://www.youtube.com/watch?v=LsF5bHRxC_M [2] https://blog.twitch.tv/en/2017/10/10/live-video-transmuxing-...

How were those "dark times" for video? At least for Real and Apple they were trying to solve then-unsolved delivery problems. Apple's video stack also covered desktop video editing/production.

Video is hard. It's a lot of data that needs to be read, processed, and displayed under tight time constraints and needs to be synchronized with associated audio playback.

This is all made more challenging if you're trying to do it on storage, bandwidth, memory, and processing constrained consumer computing hardware. Compression somewhat solved the storage and bandwidth problems but not processing. Better fidelity codecs needed a lot more processing power to decode.

In the early 90s you had MPEG-1 at the high end of the quality scale but was so processor intensive you needed decoder ASICs in consumer hardware just to play it back. Then you had codecs like Cinepak that were far less processor intensive but middling quality. Then you had much lower fidelity like Microsoft's Video1, Apple Video, and even Smacker which had very low decoding requirements but didn't look great.

Network delivery of any of those in consumer hardware was a pipe dream when 14.4K modems were still rare and 9.6k were common. The h.261 codec, which MPEG-1 was based on, had a minimum bitrate of 64kbps which was out of reach of pretty much everyone.

Besides the hardware decoding requirements of MPEG-1 it was entirely unsuited for editing. Both QuickTime and Video for Windows were meant for editing on consumer desktop machines. The codecs they supported were meant for editing and then delivery (on CD-ROM typically).

In the mid to late 90s processing power had advanced such that MPEG-1 and h.261/3 could be decoded real-time in software. RealVideo and Sorensen Video 1 were both based on drafts of the h.263 spec which included video conferencing over POTS connections in its design criteria.

Again I'm not seeing dark times for digital video. There were lots of codecs because they had different uses, limitations, and strengths. The h.26x codecs were designed for video conferencing and it was Real and to a lesser extent Apple that realized they were also useful for streaming over the internet. Both MPEG-1/2 were unsuited for streaming as they didn't support variable frame rates and handled low dial-up bitrates poorly at best. It wasn't until the MPEG-4 überspec that internet streaming, video conferencing, and disc-based delivery settled under a single specification.

While ffmpeg is an amazing project and widely used, it didn't really do anything to settle the proliferation of codecs and containers. It really was MPEG-4 that allowed for that to happen to the extent it's happened.

Props out to Michael Niedermayer, he's been maintaining ffmpeg for years and does the bulk of the heavy lifting! Thanks! https://www.linkedin.com/in/michaelniedermayer

Presence online is overrated.

Possibly more --runtime switches than any other command i use