>
On Remotion, yeah, not sure it's the right fit, but honestly the sheer capability of models at writing code these days has surprised me.Just to clarify I still think code-driven graphics is the correct approach, but in my case I opted for a different library with a more powerful imperative API.
> Also noticed they have an experimental client-side rendering version built on mediabunny
Yes, I've tried it out, it was a non-starter for me because it only supports canvas-based components, and Remotion didn't seem to have good support for text on canvas because they rely on HTML for most of that.
> On WebCodecs, there are a fair set of challenges, but we wanted to take the bet
Totally understand the appeal and immediacy of a browser app, I was lured in by that too. For what it's worth I've reported showstopping WebCodecs issues in Chromium and there's basically no indication they'll get fixed on a predictable timeline.
Another issue I ran into that I just remembered is animating text on canvas. It's basically impossible to get pixel-perfect anti-aliased text animation using a canvas. I would have to dig up the exact details but it was something to do with how browsers handle sub-pixel positioning for canvas text, so there was always some jitter when animating. This coupled with the aforementioned WebCodecs issues led me to conclude that professional-quality video rendering is not currently possible in the browser environment. Aliasing, jitter and artifacts are immediately perceptible and are the type of thing that users have zero tolerance for (speaking from experience).
This is not meant to be discouraging in any way, I've just been very deep into this rabbithole and there are some very nasty well-hidden pitfalls.