Gives a new perspective on a classic verse: "For he spoke, and it came to be; he commanded, and it stood firm." Psalm 33:9, NIV :)
How so?
DreamFusion: Text-to-3D using 2D Diffusion
121–130 of 208 posts
Re: DreamFusion: Text-to-3D using 2D Diffusion
#122Oh my god, we are done for
Re: DreamFusion: Text-to-3D using 2D Diffusion
#123Did we hit some sort of technical inflection point in the last couple of weeks or is this just coincidence that all of these ML papers around high quality procedural generation are just dropping every other day?
Partially coincidence, but also ICLR submission deadline was yesterday, so now papers can be public.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#124I'm not even going to pretend that I have a clue on how this is done. But I'm wondering if the output can be turned into 3d objects that can be used in any of the 3D modeling software? It would be a game changer in terms of real world product development in both of speed and ease.
Well they even have a “downlod model” so yep you definitely can. I wouldn’t think of this as an amazing panacea though, since once everyone has access to it that suddenly means whatever reason making assets like this was valuable before, will now be dirt cheap for all and thus actually net negative for people in that industry. Just saying and warning, not to be a bummer
FWIW, there's still a pretty big gap between a single static mesh and something that is a usable asset, say in a game. Maybe this could provide a shortcut for a modeler to get started, but it still is going to take a lot of skill from that point.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#125So does this mean I can use DreamBooth to create plausible NERFs of myself in any scenario? The future is looking weird.
Nah. This is made by Google, so it'll never become useful.
You'll have to wait a few months for someone else to replicate it.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#126Earlier quoted context omitted.
Aware of the link, though you have not provided any clarification for why there are two links; strikes me as odd if authors are trying to post it anonymously that simple Google finds authors names.
I believe ICLR guidelines require the authors to submit papers and any supplementary materials (including links to webpages, videos, etc) without identifying information, but authors are not barred from public announcements on other forums. IIUC, the idea behind this policy was originally to accommodate author freedom to engage in common practices such as simultaneous submission to arxiv (which identifies the authors…
Thanks, appreciate explanation - maybe it’s me, but found it odd, didn’t know what to make of it. Maybe it’s me, but if there not against authors posting with their identity public elsewhere, they should just have a button you’re able to click, a call warning, and then get to see the authors.
Re: DreamFusion: Text-to-3D using 2D Diffusion
#127The most incredible thing here is that this demonstrates a level of 3D understanding that I didn't believe existed in 2D image models yet. All of the 3D information in the output was inferred from the training set, which is exclusively uncurated and unsorted 2D still images. No 3D models, no camera parameters, no depth maps. No information about picture content other than a text label (scraped from the web and often…
As far as I understand from a quick read of the paper, the 2D diffusion doesn't have a 3D understanding. It probably have some sort of local neighborhood understanding, aka small geometric transformation of objects map close to each other in the diffusion space (That's why like with latent spaces you can "interpolate" ( https://replicate.com/andreasjansson/stable-diffusion-animat... ) in the diffusion space). But tha…
The model clearly has an understanding of the 3D structure of objects. If it didn't, using it to generate 3D models wouldn't work. The knowledge that the leg bone is connected to the knee bone, etc, isn't coming from NeRF, it's all in the "2D" model. Sure, maybe you could distill that knowledge into a different model architecture that is somehow natively 3D in order to improve the efficiency of sampling. But that's more or less just an optimization. What's interesting is the fact that the knowledge is in there already, learned without 3D input data.
Humans don't get natively 3D training data either. We don't have depth sensors or any other natively 3D senses. Our eyes only ever see 2D images and we learn 3D structure from that, so I guess these 2D image models are doing something analagous. Except they don't even have the benefit of stereo images!
Re: DreamFusion: Text-to-3D using 2D Diffusion
#128Earlier quoted context omitted.
"You're posting too fast" is a limit that's manually applied to accounts that have a history of "posting too many low-quality comments too quickly and/or getting into flame wars". You can email dang (hn@ycombinator.com) if you think it was applied in error, but if it's been applied to you more than once... you probably have a continuing problem with proliferating flame wars or posting otherwise combative comments.
I think you also can get it by just having unpopular opinions. Hackernews used to be much more nuanced than it is today imo.
Uhm... did you even check my account age ((and my old one is two years older))
Re: DreamFusion: Text-to-3D using 2D Diffusion
#129Huh, it's a pretty similar technique to what I outlined a couple days ago: https://news.ycombinator.com/item?id=32965139 Although they start with random initialization and a text prompt. It seems to work well. I now see no reason we can't start with image initialization!
directly training a NeRF on a single image is a terribly unconstrained problem that would lead to a volume that looks bad when the viewpoint changes. the gist of render + use diffusion model to refine is a great idea though and core to our method! the details of how to use the diffusion model for this refinement was the challenge, but once we figured that out it... just worked :)
Re: DreamFusion: Text-to-3D using 2D Diffusion
#130As someone who went to college for 3D animation in +*1997*+ AND DESIGNED the datacenter for luca' presidio complex..
where-by learning that Pixar was developed by steve jobs when lucas didnt think there was a future for computer animation... and so steve bought the death star from lucas...
That became pixar...
AI is going to fucking kill it - what will happen in the next decade will be ANYONE uploading a script to an AI to make a full length movie...
AND their will be editing tools as well that are AI driven...
Like mentioned by William Gibson
*The future is here, its just not evenly distributed yet*