Is multimedia programming inherently hard?

>What doesn't work? Why doesn't it work? Is the source of the problems in code or in the libraries?

>Due to concurrency issues or?

It's difficult to name one but probably the biggest issue at the moment is frame rates randomly drop at some point the the resultant video is garbled (1 minute is played in a few seconds, etc.)

Possibly concurrency is one of the reasons. The Media Foundation encoder expects video and audio frames at regular intervals and if because of performance issues, some frames drop, it gets into a vicious cycle and frame drops accelerate until it comes to stand still.

>You're not a bad programmer - maybe you're just bad at system design/software architecture.

Unfortunately, I think I am because I suck at algorithmic challenges (competitive programming), system design, reading documentation for external APIs, in short nearly everything that is to programming. I learned programming at 13, it goes to show I have been programming for quite some time and others who started it in university went to do amazing things while I suck at basics. I dreamed of starting a tech startup someday but it seems unlikely now.

>figure out how to rewrite it into V3

My client is a SOB and doesn't appreciate my efforts nor rewards them, so it's unlikely. Besides, I have signed NDA so starting open source project is out of the question. I'm thinking of starting some other projects, though.

>Imagine if Picasso quit after his first sucky drawing. Maybe even HN wouldn't be here.

I liked that lowkey compliment but I doubt I am worthy of this.

Capturing screen/webcams and adding extra layers can get really complex. If you can't simulate and easily test it, it's a recipe for disaster.

Using MF API is quite old (relies on DirectX 9 as far as I remember).

C++/COM/multi-media is not such a good match - I always felt like puking when having to deal with COM interfaces.

I would strongly suggest you switch to C# (note: I've programmed 13 years in C++, and the last 9 in C#). There's pitfalls in this too - namely, not sure if/how easy you can easily do multimedia without UWP (Universal Windows Platform). That's a big can of worms, really hard to grasp, big learning curve, but well worth it in time (if you want to see what you can achieve, you can take a quick look at www.phot-awe.com - that's UWP).

Having said all the above - I highly suggest C#/UWP, but DO NOT expect an easy ride. It will take quite a while to be productive, but that will be time well spent.

First of all, doubting your own competence is a good sign. Especially combined with a willingness to ask questions and act on feedback. I'd be extremely skeptical hiring someone who had no self-doubt.

It sounds like you just tackled too many novel things at the same time:

- 160h, 7000 line project is a very large project for a student. Managing software complexity is its own skill.

- Multithreaded programming is hard.

- Performant, low-latency programming is hard.

- "Systems" programming is hard.

- Video capture, codecs, etc. are a narrow niche. Microsoft COM API's are not exactly the simplest things...

- C++ is a particularly unforgiving language. It doesn't help that the language is so versatile that 90's C++ libraries/tutorials/etc look nothing like "modern" C++ from the 00's, which was itself obsoleted by more "modern" C++ from the 10's...

In terms of advice, I get the impression that you're not looking forward to writing a third version of this project, so here are a few generic tips for whatever you choose to do next:

- It sounds like you wrote the code that does whatever the application is supposed to do, but didn't write the supporting tests/tooling/etc. For example, elsewhere you mention that you think the application might be dropping frames due to this or that, but you're not sure. The next step--and honestly you need to do this preemptively even before there are any problems--is to add very detailed logging to provide you with hard data. Furthermore, in an audiovisual application like this, you'll probably have to write specialized tools to parse and analyze those logs graphically since they'll be dumping a tremendous amount of data to disk. Don't be afraid to write more test and tool code than main application code.

- Sometimes it's worth it to take some extra time to perform "studies"--code that you write to experiment with something, then throw away. For example, elsewhere in this thread you asked if it would be better to buffer 10 frames at a time or to process frames synchronously. Well, this is a great opportunity to do some "science": think about what you expect will happen, fork the code to a new branch in source control, implement the change in quick-and-dirty way, and check the results. If it confirmed your hypothesis and helped, switch back to your original branch and re-implement it in a clean way. If it confirmed your prediction and didn't help, switch back to your original branch and just make a note of this experiment in the documentation. Negative results like this are important since they can reduce the surface area of things you need to troubleshoot later. If the result was surprising in some sense, investigate further and if post to IRC or StackOverflow if you get stuck.

- Regarding "solving hard problems". Forget about how hard or easy something is in an absolute sense. I wouldn't even worry too much about comparing yourself with peers. Just worry about self improvement. And the best way to improve yourself is to introduce a little bit of novelty at a time. For example, if you're learning a new language, re-implement some toy application you've already written before in C++. Don't try to learn a new language while also introducing known-hard things like networking, concurrency, etc.