Learnings from 4 months of Image-Video VAE experiments
1–10 of 19 posts
Re: Learnings from 4 months of Image-Video VAE experiments
#2Hi HN, I’m one of the two authors of the post and the Linum v2 text-to-video model (https://news.ycombinator.com/item?id=46721488). We're releasing our Image-Video VAE (open weights) and a deep dive on how we built it. Happy to answer questions about the work!
Re: Learnings from 4 months of Image-Video VAE experiments
#3[flagged]
Re: Learnings from 4 months of Image-Video VAE experiments
#4This seems like a great model to experiment fine tuning with original art, given it’s relatively small and with open license. Is that a fair assessment?
Thanks for the great write up and making it available to us all.
Re: Learnings from 4 months of Image-Video VAE experiments
#5This seems like a great model to experiment fine tuning with original art, given it’s relatively small and with open license. Is that a fair assessment? Thanks for the great write up and making it available to us all.
yep, Apache 2.0! so anyone's welcome to download and hack away
Re: Learnings from 4 months of Image-Video VAE experiments
#6Nice summary! I missed the mention of EQ-VAE when it comes to generation quality. Tiny trick, huge impact! Have you tried it?
Re: Learnings from 4 months of Image-Video VAE experiments
#7Hi HN, I’m one of the two authors of the post and the Linum v2 text-to-video model ( https://news.ycombinator.com/item?id=46721488 ). We're releasing our Image-Video VAE (open weights) and a deep dive on how we built it. Happy to answer questions about the work!
No questions but I appreciate the write-up! Thank you for sharing.
Re: Learnings from 4 months of Image-Video VAE experiments
#8Nice summary! I missed the mention of EQ-VAE when it comes to generation quality. Tiny trick, huge impact! Have you tried it?
Hadn’t seen that before! Seems very in line with what with the broader points about regularization. In table 4 they show faster convergence in 200 epochs when used alongside REPA. I’d be curious to see if it ended up beating REPA by itself with full 800 epochs of training — or if something about this new latent space, leads to plateauing itself (learns faster but caps out on expressivity). We’ve seen that phenomena before in other situations (eg UNET learns faster than DiT because of convolutions, but stops learning beyond a certain point).
Re: Learnings from 4 months of Image-Video VAE experiments
#9Very nice well written article!
The kind that I like so much on HN. It tickle your mind but is still clear enough for an advanced beginner.
Re: Learnings from 4 months of Image-Video VAE experiments
#10its cool to see the iterative improvements to your model laid out, but for everything that workedm i imagine there were at least a million other things you also tried but didnt work out. whats your process of trying these different techniques/architectures? do you just wait for one experiment to finish and visually inspect the results everytime. seems hard since these take a while to train. how do you shorten the feedback loop in this space?