Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing an MP4 Muxer for Fun and Profit

obsproject.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Writing an MP4 Muxer for Fun and Profit

#11

Post by 38 »

karolist wrote: post #3

> Except there is no profit, only pain I have 20 years or professional experience and my conclusion, if someone asked, what IT boils down to: pain. The pain is what filters who can succeed and who fail. Can you endure hunting a bug for 7 hours in your chair? Can you fix problem after problem to get a system running? Everything that can fail, will fail, and you have to deal with it.

Or when you encounter (another) corner case that requires a top down rewrite of your code. If you start to question your sanity or the meaning of life, you're probably on the right track.

Re: Writing an MP4 Muxer for Fun and Profit

#13

Post by der_rod »

Retr0id wrote: post #4

> It kind of hurts that several days of work and research can be summed up in a couple paragraphs, but that's what the "pain" part in the subtitle is for. Having recently written my own fragmented-MP4 remuxing library, I felt this pain too, and my soon-to-be-published writeup has very similar things to say about the ISO's paywalling practices. I think one of the hardest parts of ISO-BMFF, aside from spec availability…

> Having recently written my own fragmented-MP4 remuxing library, I felt this pain too, and my soon-to-be-published writeup has very similar things to say about the ISO's paywalling practices.

Would be curious to hear what goals you had with writing a muxer yourself as well, given that most people just use LibAV/GStreamer/GPAC and call it a day.

> I think one of the hardest parts of ISO-BMFF, aside from spec availability, is that it's pretty hard to implement "cleanly", making existing code confusing to use as reference. (My own implementation is certainly not clean either)

I certainly wouldn't call the OBS implementation "clean" either. It's very much inspired by the FFmpeg/LibAV implementation since that one is fairly straightforward (not a lot of abstraction), and gets the job done (and also is GPL/LGPL so not a huge concern looking at it).

Re: Writing an MP4 Muxer for Fun and Profit

#14

Post by CrendKing »

Can someone explain how does an existing media player understand the new mdat format without modification? I assume if they find a completed moov at end of the file, it would recognize the file as a unfragmented mp4. It should then try to find a list of recognized codecs directly inside the mdat (like in the first picture), but instead they will find another moov, a bunch of moofs and sub-mdats, all of which are clearly not proper for a unfragmented mp4. Why doesn't the player report this as a "unrecognizable, badly formatted" mp4 file?

Re: Writing an MP4 Muxer for Fun and Profit

#15

Post by Andrews54757 »

Having worked with some MP4 demuxing for my extension [1], I feel the pain. Lots of times I would play the video only to find inexplicable issues such as drifting audio. I highly recommend using an mp4 inspector tool, such as mp4box [2], to debug these issues.

1: https://github.com/Andrews54757/FastStream

2: https://gpac.github.io/mp4box.js/test/filereader.html

Re: Writing an MP4 Muxer for Fun and Profit

#16

Post by Andrews54757 »

CrendKing wrote: post #14

Can someone explain how does an existing media player understand the new mdat format without modification? I assume if they find a completed moov at end of the file, it would recognize the file as a unfragmented mp4. It should then try to find a list of recognized codecs directly inside the mdat (like in the first picture), but instead they will find another moov, a bunch of moofs and sub-mdats, all of which are clea…

The moov contains a list of byte offsets which the player can use to directly access media data. You can skip the moofs and other headers inside by using gaps in the offsets.

Re: Writing an MP4 Muxer for Fun and Profit

#17

Post by Retr0id »

der_rod wrote: post #13
Retr0id wrote: post #4

> It kind of hurts that several days of work and research can be summed up in a couple paragraphs, but that's what the "pain" part in the subtitle is for. Having recently written my own fragmented-MP4 remuxing library, I felt this pain too, and my soon-to-be-published writeup has very similar things to say about the ISO's paywalling practices. I think one of the hardest parts of ISO-BMFF, aside from spec availability…

> Having recently written my own fragmented-MP4 remuxing library, I felt this pain too, and my soon-to-be-published writeup has very similar things to say about the ISO's paywalling practices. Would be curious to hear what goals you had with writing a muxer yourself as well, given that most people just use LibAV/GStreamer/GPAC and call it a day. > I think one of the hardest parts of ISO-BMFF, aside from spec availabi…

The short answer is, it's for an exploit. It involves some slightly less-well-trodden boxes, and adding specially crafted metadata to live-generated videos in real-time, which existing libraries couldn't help me with much (and I did spend some time fighting a few libraries, but couldn't make them do precisely what I wanted).

"Library" is perhaps an overstatement, it does the things I need and not much more.

Re: Writing an MP4 Muxer for Fun and Profit

#18

Post by Retr0id »

Andrews54757 wrote: post #15

Having worked with some MP4 demuxing for my extension [1], I feel the pain. Lots of times I would play the video only to find inexplicable issues such as drifting audio. I highly recommend using an mp4 inspector tool, such as mp4box [2], to debug these issues. 1: https://github.com/Andrews54757/FastStream 2: https://gpac.github.io/mp4box.js/test/filereader.html

I found this inspector useful, too: https://mlynoteka.mlyn.org/mp4parser/

Re: Writing an MP4 Muxer for Fun and Profit

#19

Post by lylejantzi3rd »

karolist wrote: post #3

> Except there is no profit, only pain I have 20 years or professional experience and my conclusion, if someone asked, what IT boils down to: pain. The pain is what filters who can succeed and who fail. Can you endure hunting a bug for 7 hours in your chair? Can you fix problem after problem to get a system running? Everything that can fail, will fail, and you have to deal with it.

I came to the same conclusion in business. Is what you're doing 100% pure pain? Good. It means you're providing value.

Re: Writing an MP4 Muxer for Fun and Profit

#20

Post by der_rod »

Retr0id wrote: post #17
der_rod wrote: post #13

Earlier quoted context omitted.

> Having recently written my own fragmented-MP4 remuxing library, I felt this pain too, and my soon-to-be-published writeup has very similar things to say about the ISO's paywalling practices. Would be curious to hear what goals you had with writing a muxer yourself as well, given that most people just use LibAV/GStreamer/GPAC and call it a day. > I think one of the hardest parts of ISO-BMFF, aside from spec availabi…

The short answer is, it's for an exploit. It involves some slightly less-well-trodden boxes, and adding specially crafted metadata to live-generated videos in real-time, which existing libraries couldn't help me with much (and I did spend some time fighting a few libraries, but couldn't make them do precisely what I wanted). "Library" is perhaps an overstatement, it does the things I need and not much more.

A hah, that sounds cool, looking forward to the writeup!
Post reply on HN