Did we hit some sort of technical inflection point in the last couple of weeks or is this just coincidence that all of these ML papers around high quality procedural generation are just dropping every other day?
[deleted]
DreamFusion: Text-to-3D using 2D Diffusion
21–30 of 208 posts
Re: DreamFusion: Text-to-3D using 2D Diffusion
#22Did we hit some sort of technical inflection point in the last couple of weeks or is this just coincidence that all of these ML papers around high quality procedural generation are just dropping every other day?
From the abstract: “We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss.”
This seems like basically plugging a couple of techniques together that already existed, allowing to turn 2D text-to-image into 3D text-to-image.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#23@dang The link should be updated to https://dreamfusion3d.github.io
Re: DreamFusion: Text-to-3D using 2D Diffusion
#24This is crazy good - most prior text-to-3d models produced weird amorphous blobs that would kind of look like the prompt from some angles, but had no actual spatial consistency.
Blown away by how quickly this stuff is advancing, even as someone who's relatively cynical about AI art.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#25Amazing! How long then until we get photorealistic AI generated 3D VR games and experiences in the metaverse?
Re: DreamFusion: Text-to-3D using 2D Diffusion
#26This sounds like something that could be made to work with stable diffusion if someone just implements the code based on the paper.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#27Did we hit some sort of technical inflection point in the last couple of weeks or is this just coincidence that all of these ML papers around high quality procedural generation are just dropping every other day?
From the abstract: “We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss.” This seems like b…
Time and time again these ML techniques are proving to be wildly modular and pluggable. Maybe sooner or later someone will build a framework for end to end text-to-effective-ML-architecture that will just plug different things together and optimize them.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#28It's funny that the authors are 'anonymous' but they have access to Imagen so obviously it's by Google.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#29Did we hit some sort of technical inflection point in the last couple of weeks or is this just coincidence that all of these ML papers around high quality procedural generation are just dropping every other day?
From the abstract: “We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss.” This seems like b…
> This seems like basically plugging a couple of techniques together that already existed
as with a majority of ML research
Re: DreamFusion: Text-to-3D using 2D Diffusion
#30It's funny that the authors are 'anonymous' but they have access to Imagen so obviously it's by Google.
[deleted]