Text-to-4D Dynamic Scene Generation
make-a-video3d.github.io
Text-to-4D Dynamic Scene Generation
1–10 of 19 posts
Re: Text-to-4D Dynamic Scene Generation
#2Link to paper https://arxiv.org/abs/2301.11280, dynamic visualisations only work in Chrome (?)
Re: Text-to-4D Dynamic Scene Generation
#3Link to paper https://arxiv.org/abs/2301.11280 , dynamic visualisations only work in Chrome (?)
Can confirm it doesn’t work on brave on mobile
Re: Text-to-4D Dynamic Scene Generation
#4I've expected NERF + Diffusion models for a while, but it looks like there's still a lot of work needed before it gets practical.
Re: Text-to-4D Dynamic Scene Generation
#5I've expected NERF + Diffusion models for a while, but it looks like there's still a lot of work needed before it gets practical.
Performing these optimization processes during inference time has never been very practical for generative tasks, as it requires a lot of time, memory (to store the gradient) and the quality is usually mediocre. I still remember VQGAN+CLIP, the optimization process was to find a latent embedding that would maximize the cosine similarity between the CLIP encoded image and the CLIP encoded prompt, It worked but not very practical.
Re: Text-to-4D Dynamic Scene Generation
#6> trained only on Text-Image pairs and unlabeled videos
This is fascinating. It's able to pick up sufficiently on the fundamentals of 3D motion from 2D videos, while only needing static images with descriptions to infer semantics.
Re: Text-to-4D Dynamic Scene Generation
#7Getting something that generates multiple angles of the same subject in different typical poses would go a long way.
I can get midjourney to kind of do this by asking for "multiple angles", but it's hit or mis.
Re: Text-to-4D Dynamic Scene Generation
#8Can someone explain what’s 4D about this? Is it 4D because the 3D models are animated (moving)?
Re: Text-to-4D Dynamic Scene Generation
#9Can someone explain what’s 4D about this? Is it 4D because the 3D models are animated (moving)?
4D: Height, width, depth, and time.
Re: Text-to-4D Dynamic Scene Generation
#10Here we go again. The samples look uncannily similar to the early text-to-image stuff we had.