Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Video Diffusion

stability.ai

Post reply on HN
Print view View original

281–290 of 316 posts

Re: Stable Video Diffusion

#281

Post by JCharante »

4death4 wrote: post #116
alberth wrote: post #107

Earlier quoted context omitted.

What was the big “unlock” that allowed so much progress this past year? I ask as a noob in this area.

I think these are the main drivers behind the progress: - Unsupervised learning techniques, e.g. transformers and diffusion models. You need unsupervised techniques in order to utilize enough data. There have been other unsupervised techniques in the past, e.g. GANs, but they don't work as well. - Massive amounts of training data. - The belief that training these models will produce something valuable. It costs betwe…

> The belief that training these models will produce something valuable

Exactly. The growth in the next decade is going to be unimaginable because now governments and MNCs believe that there realistically be progress made in this field.

Re: Stable Video Diffusion

#282

Post by JCharante »

Cyphase wrote: post #120
alberth wrote: post #107

Earlier quoted context omitted.

What was the big “unlock” that allowed so much progress this past year? I ask as a noob in this area.

One factor is that Stable Diffusion and ChatGPT were released within 3 months of each other – August 22, 2022 and November 3, 2022, respectively. That brought a lot of attention and excitement to the field. More excitement, more people, more work being done, more progress. Of course those two releases didn't fall out of the sky.

Dalle 2 also went viral around the same time

Re: Stable Video Diffusion

#283

Post by btbuildem »

TacticalCoder wrote: post #52
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

That indeed sounds like a very plausible solution -- working with AI on the level of scene definitions, model geometries etc.

However, 3D is just one approach to rendering visuals. There are so many other styles and methods how people create images, and if I understand correctly, we can do image-to-text to analyze image content, as well as text-to-image to generate it - regardless of the orginal method (3d render or paintbrush or camera lens). There are some "fuzzy primitives" in the layers there that translate to the visual elements.

I'm hoping we see "editors" that let us manipulate / edit / iterate over generated images in terms of those.

Re: Stable Video Diffusion

#284

Post by btbuildem »

01100011 wrote: post #204
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

I recently tried to generate clip art for a presentation using GPT-4/DALL-E 3. I found it could handle some updates but the output generally varied wildly as I tried to refine the image. For instance, I'd have a cartoon character checking its watch and also wearing a pocket watch. Trying to remove the pocket watch resulted in an entirely new cartoon with little stylistic continuity to the first. Also, I originally tr…

Yes that's exactly what I'm referring to! It feels as if there is no context continuity between the attempts.

Re: Stable Video Diffusion

#285

Post by btbuildem »

achileas wrote: post #215
Zetaphor wrote: post #100

Earlier quoted context omitted.

This has become a colloquial term for describing all sports, not the insult you're perceiving it to be. Rather than projecting your own hangups and calling people names, try instead assuming that they're not trying to offend you personally and are just using common vernacular.

If only there was an existing way to refer to sports generally! And OP was referring to a specific sport (baseball), not sports generally.

The Rogers Centre hosts baseball, football, and basketball games - so in this case "sportsball" was just a shorthand for all these ball sports.

Re: Stable Video Diffusion

#286

Post by whywhywhywhy »

TacticalCoder wrote: post #52
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

Nah I disagree, this feels like a glorification of the process not the end result. Just because having the 3D model in the scene with all the lighting makes the end result feel more solid to you because you feel you can see the work that's going into it.

In the end diffusion technology can make a more realistic image faster than a rendering engine can.

I feel pretty strongly that this pipeline will be the foundation for most of the next decade of graphics and making things by hand in 3D will become extremely niche because lets face it anyone who has worked in 3D it's tedious, it's time consuming, takes large teams and it's not even well paid.

The future is just tools that give us better controls and every frame will be coming from latent space not simulated photons.

I say this as someone who had done 3D professionally in the past.

Re: Stable Video Diffusion

#287

Post by btbuildem »

achileas wrote: post #216
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. Nearly all of the available models have this, even the highly commercialized ones like in Adobe Firefly and Canva, it’s called inpainting in most tools.

I think that's more "inpainting" where the existing software solution uses AI to accelerate certain image editing tasks. I was looking for whole-image manipulation at the "conceptual" level.

Re: Stable Video Diffusion

#288

Post by btbuildem »

dsmmcken wrote: post #225
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

Adobe is doing some great work here in my opinion in terms of building AI tools that make sense for artist workflows. This "sneak peak" demo from the recent Adobe Max conference is pretty much exactly what you described, actually better because you can just click on an object in the image and drag it. See video: https://www.adobe.com/max/2023/sessions/project-stardust-gs6...

Right, that's embedded directly into the existing workflow. Looks like a very powerful feature indeed.

Re: Stable Video Diffusion

#289

Post by PawgerZ »

coldtea wrote: post #268
COAGULOPATH wrote: post #240

Earlier quoted context omitted.

Of course they would. The internet spent a solid month laughing at the Sonic the Hedgehog movie because Sonic had weird-looking teeth.

Since that movie did well and spawned 2 sequels, the real conclusion is that the viewers didn't really care. As for "the internet", there will always some small part of it which will obsess and/or laught over anything, doesn't mean they represent anything significant - not even when they're vocal.

Viewers did care: the teeth got changed before the movie was released. And, I don't know if you missed it, but it wasn't just one niche of the internet commenting on his teeth. The "outrage" went mainstream; even dentists were making hit-pieces on Sonic's teeth. I'm not gonna lie, it was amazing marketing for the movie, intentional or not.

Re: Stable Video Diffusion

#290

Post by iamgopal »

Very soon, we will be able to change story line of a web series dynamically, a little more thrill, a little more comedy, changing character face to matching ours and others, all in 3D with 360 degree view, how far are we from this ? 5 year ?
Post reply on HN