Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DreamFusion: Text-to-3D using 2D Diffusion

dreamfusion3d.github.io

Post reply on HN
Print view View original

151–160 of 208 posts

Re: DreamFusion: Text-to-3D using 2D Diffusion

#151

Post by eurasiantiger »

ecmascript wrote: post #107
LordDragonfang wrote: post #100

Earlier quoted context omitted.

"You're posting too fast" is a limit that's manually applied to accounts that have a history of "posting too many low-quality comments too quickly and/or getting into flame wars". You can email dang (hn@ycombinator.com) if you think it was applied in error, but if it's been applied to you more than once... you probably have a continuing problem with proliferating flame wars or posting otherwise combative comments.

I think you also can get it by just having unpopular opinions. Hackernews used to be much more nuanced than it is today imo.

It’s really easy to get throttled for a single comment out of thousands.

Re: DreamFusion: Text-to-3D using 2D Diffusion

#152

Post by dr_dshiv »

AStrangeMorrow wrote: post #58
dr_dshiv wrote: post #52

Earlier quoted context omitted.

It’s called the technological singularity. Pretty fun so far!

This isn't what is usually meant by "technological singularity". It is an inflection point where technology growth becomes incontrollable and unpredictable, usually theorized to be cause by a self improving agent (/AI) that becomes smarter with each of its iterations. This is still standard technological progress, human control, even if very fast

It’s basically when AI starts self-improving. I think this started with large language models. They are central to these developments. Complete autonomy is not required for AGI-nor therefore for the singularity.

Whatever it is, this is a massive phase shift.

Re: DreamFusion: Text-to-3D using 2D Diffusion

#153

Post by jillesvangurp »

beambot wrote: post #57
layer8 wrote: post #22

Earlier quoted context omitted.

From the abstract: “We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss.” This seems like b…

> This seems like basically plugging a couple of techniques together that already existed [...] In his Lex Fridman interview, John Carmack makes similar assertions about this prospect for AGI: That it will likely be the clever combination of existing primitives (plus maybe a couple novel new ones) that make the first AGI feasible in just a couple thousand lines of code.

That was a great interview. I really liked his perspective on how close we are to having AGI. His point is that there's only a few more things we need to figure out and then it will basically happen.

I also liked the analogy he made with his earlier work on 2D and 3D graphics engines where taking a few short cuts basically got him on a path to success. For a while we had this "almost" 3D capability long before the hardware was ready to do 3D properly. It's the same with AGIs. A few short cuts will get us AI that is pretty decent and can do some impressive things already - as witnessed by the recent improvements in image generation. It's not a general AI but it has enough intelligence that it can still do photo realistic images that make sense to us. There's a lot of that happening right now and just scaling that up is going to be interesting by itself.

Re: DreamFusion: Text-to-3D using 2D Diffusion

#154

Post by LarsDu88 »

As someone who dabbles in 3d modeling, this is going to be an incredible resource for creating static 3d objects. Someone ought to come up with a way to convert to mesh better than the Marching Cubes algorithm I've seen applied to most NERFs. The models still lack coherent topology and would probably be janky if fully rigged.

Re: DreamFusion: Text-to-3D using 2D Diffusion

#155

Post by aliswe »

drKarl wrote: post #40
drKarl wrote: post #25

Amazing! How long then until we get photorealistic AI generated 3D VR games and experiences in the metaverse?

Why the downvote? I wasn't being sarcastic, it was a honest question, I'm really impressed how far this technology has come since GPT-3 2 years ago to DALl-E and Stable Diffusion ro Meta's text to video to this...

I'm guessing the post could be interpreted as "normie" and not HN curiosity ;)

Re: DreamFusion: Text-to-3D using 2D Diffusion

#156

Post by LarsDu88 »

cuuupid wrote: post #74
naillo wrote: post #28

It's funny that the authors are 'anonymous' but they have access to Imagen so obviously it's by Google.

A large portion of the ML community (rightly) discredits Google papers because: - they rarely provide the data or code used so it's basically "i swear it works bro" research - what they achieve is usually through having the most pristine dataset on the planet and is often unusable by other researchers - other times they publish papers that are basically "we slightly modified this excellent open source paper, slapped…

IMO the biggest algorithmic advances made by Google such as the transformer have greatly pushed the field forward. The giant model's that will have similar variations released in the next 3 months actually aren't that important on a conceptual level except as a PoC.

Re: DreamFusion: Text-to-3D using 2D Diffusion

#157

Post by poolio »

LarsDu88 wrote: post #154

As someone who dabbles in 3d modeling, this is going to be an incredible resource for creating static 3d objects. Someone ought to come up with a way to convert to mesh better than the Marching Cubes algorithm I've seen applied to most NERFs. The models still lack coherent topology and would probably be janky if fully rigged.

With smooth enough geometry converting NeRFs to meshes with marching cubes works pretty well. Would you say the topology of meshes on our website are still too incoherent for rigging?

Re: DreamFusion: Text-to-3D using 2D Diffusion

#159

Post by etaioinshrdlu »

bmpoole wrote: post #129
etaioinshrdlu wrote: post #4

Huh, it's a pretty similar technique to what I outlined a couple days ago: https://news.ycombinator.com/item?id=32965139 Although they start with random initialization and a text prompt. It seems to work well. I now see no reason we can't start with image initialization!

directly training a NeRF on a single image is a terribly unconstrained problem that would lead to a volume that looks bad when the viewpoint changes. the gist of render + use diffusion model to refine is a great idea though and core to our method! the details of how to use the diffusion model for this refinement was the challenge, but once we figured that out it... just worked :)

Very cool! Can we tweak your algorithm a little to be seeded with a real photo from a single viewpoint?

Re: DreamFusion: Text-to-3D using 2D Diffusion

#160

Post by sandgiant »

modeless wrote: post #127
GistNoesis wrote: post #101

Earlier quoted context omitted.

As far as I understand from a quick read of the paper, the 2D diffusion doesn't have a 3D understanding. It probably have some sort of local neighborhood understanding, aka small geometric transformation of objects map close to each other in the diffusion space (That's why like with latent spaces you can "interpolate" ( https://replicate.com/andreasjansson/stable-diffusion-animat... ) in the diffusion space). But tha…

The model clearly has an understanding of the 3D structure of objects. If it didn't, using it to generate 3D models wouldn't work. The knowledge that the leg bone is connected to the knee bone, etc, isn't coming from NeRF, it's all in the "2D" model. Sure, maybe you could distill that knowledge into a different model architecture that is somehow natively 3D in order to improve the efficiency of sampling. But that's m…

I think this is a misunderstanding of how these models work. The model does not understand anything at all. It's computing correlations. I could spend all day computing correlation without ever understanding what the correlations correspond to in the physical world. The correlations could still amount to a useful description of some physical phenomenon, but their interpretation requires much more than just the ability to compute correlations in some limited space.

If we elevate "correlations" to mean "understanding", we're quickly going to run out of words.

Post reply on HN