Live data from Hacker News

SPAD: Spatially Aware Multiview Diffusers

yashkant.github.io

21–30 of 37 posts

Re: SPAD: Spatially Aware Multiview Diffusers

#21
post #20

or Single Photon Avalanche Diode, coming to a LIDAR near you very soon if not already. Yay ambiguous acronyms.

Fortsense FL6031 - Automotive ready. For anyone not familiar with SPAD (Single Photon Avalanche Diode) YouTube it. Very impressive computational imagery through walls, around corners and such.

Re: SPAD: Spatially Aware Multiview Diffusers

#22

I’m confused why there is so much focus on text to images and models. If you spent five minutes talking to anyone with artistic ability, they would tell you that this is not how they generate their work. Making images involves entirely different parts of reasoning than that for speech and language. We seem to be building an entirely faulty model of image generation (outside of things like ControlNet) on the premise t…

Can you share some of what you have found about the creative process by talking to people with artistic ability ?

What are your ideas about the differences between a human and AI's creative process ?

Are there any similarities, or analagous processes ?

Do you think creators have an kind of latent space where different concepts are inspired by multi-modal inputs ( what sparks inspiration ? e.g. sometimes music or a mood inspires a picture ) and then the creators make different versions of their idea by combining different amounts of different concepts ?

I am not being snarky, I am genuinely interested in views comparing human an AI's creative processes.

Re: SPAD: Spatially Aware Multiview Diffusers

#24

No major comment other than this tech is obviously going to transform gaming.

to my understanding it produces 2D images (from various angles), not 3D models… But sure, it's very close to producing a 3D model.

MVDream can be combined with a NeRF (or equivalent spatial interpolator such as InstantNGP/TorchNGP) followed by marching cubes to extract a mesh and produce a 3D model. SDS loss and guidance is done from multiple equally-spaced views simultaneously. MVDream has a repo that implements this:

https://github.com/bytedance/MVDream-threestudio

It should be fairly straightforward to adapt this repo (which is based on threestudio) to use SPAD instead of MVDream.

Re: SPAD: Spatially Aware Multiview Diffusers

#25

No major comment other than this tech is obviously going to transform gaming.

to my understanding it produces 2D images (from various angles), not 3D models… But sure, it's very close to producing a 3D model.

Are there any NERF to textured or vertex coloured polygon mesh point cloud tools or papers ?

I.e. can these be assets for traditional game engines ?

Could a sort of photogrammetry like Meshroom or Reality capture do the trick ?

Re: SPAD: Spatially Aware Multiview Diffusers

#26
post #22

I’m confused why there is so much focus on text to images and models. If you spent five minutes talking to anyone with artistic ability, they would tell you that this is not how they generate their work. Making images involves entirely different parts of reasoning than that for speech and language. We seem to be building an entirely faulty model of image generation (outside of things like ControlNet) on the premise t…

Can you share some of what you have found about the creative process by talking to people with artistic ability ? What are your ideas about the differences between a human and AI's creative process ? Are there any similarities, or analagous processes ? Do you think creators have an kind of latent space where different concepts are inspired by multi-modal inputs ( what sparks inspiration ? e.g. sometimes music or a mo…

I used to work as an illustrator. Most images appeared to me as somewhere between fuzzy or clear image concepts, unaccompanied by any words. I then have to take these concepts and translate them using principles of design, color, composition, abstraction etc., such that they’re coherent and understandable to others.

Most illustration briefs are also not wrote descriptions of images because people are remarkably bad at describing what they want in an image, beyond in the most general sense of its subject. This is why you see DALLE doing all kinds of prompt elaboration on user inputs to generate “good” images. Typically, the illustrator is given the work to be illustrated (e.g. an editorial), distills key concepts from the work and translates these into various visual analogues, such as archetypes, metaphors and themes. Depending on the subject, one may have to include reference images or other work in a particular style, if the client has something specific in mind.

Re: SPAD: Spatially Aware Multiview Diffusers

#27
post #12

I’m confused why there is so much focus on text to images and models. If you spent five minutes talking to anyone with artistic ability, they would tell you that this is not how they generate their work. Making images involves entirely different parts of reasoning than that for speech and language. We seem to be building an entirely faulty model of image generation (outside of things like ControlNet) on the premise t…

Project briefs to an artist typically contain both text and reference images. Image diffusion models and the like likewise typically use a text prompt together with optional reference images.

Project briefs are generally not descriptions of images and reference tends to be more style than content focused. Source: I used to be an illustrator for major media outlets like the NYTimes, etc.

Re: SPAD: Spatially Aware Multiview Diffusers

#28
post #12

Earlier quoted context omitted.

Project briefs to an artist typically contain both text and reference images. Image diffusion models and the like likewise typically use a text prompt together with optional reference images.

Project briefs are generally not descriptions of images and reference tends to be more style than content focused. Source: I used to be an illustrator for major media outlets like the NYTimes, etc.

So how is the desired content communicated to the artist?

(Also, reference images can absolutely be used to communicate style to a diffusion model)

Re: SPAD: Spatially Aware Multiview Diffusers

#29
post #28

Earlier quoted context omitted.

Project briefs are generally not descriptions of images and reference tends to be more style than content focused. Source: I used to be an illustrator for major media outlets like the NYTimes, etc.

So how is the desired content communicated to the artist? (Also, reference images can absolutely be used to communicate style to a diffusion model)

The artist is given the content to be illustrated, extrapolates themes and overarching rhetorical or narrative aspects of the work, creates visual representations or metaphors corresponding to these aspects, generates 3-5 interpretations, shows them to the AD, who provides feedback on what has been extrapolated, as well as various design considerations.

Re: SPAD: Spatially Aware Multiview Diffusers

#30
post #13

I’m confused why there is so much focus on text to images and models. If you spent five minutes talking to anyone with artistic ability, they would tell you that this is not how they generate their work. Making images involves entirely different parts of reasoning than that for speech and language. We seem to be building an entirely faulty model of image generation (outside of things like ControlNet) on the premise t…

Check out invoke.ai for an example of something much closer to a professional tool.

nah, that has absolutely nothing to do with what they're saying. I've used it for over a year and this is a weird way for it to appear in conversation, I hope you're not astroturfing
Post reply on HN