Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Video Diffusion

stability.ai

Post reply on HN
Print view View original

241–250 of 316 posts

Re: Stable Video Diffusion

#241

Post by valec »

mkaic wrote: post #130
4death4 wrote: post #116

Earlier quoted context omitted.

I think these are the main drivers behind the progress: - Unsupervised learning techniques, e.g. transformers and diffusion models. You need unsupervised techniques in order to utilize enough data. There have been other unsupervised techniques in the past, e.g. GANs, but they don't work as well. - Massive amounts of training data. - The belief that training these models will produce something valuable. It costs betwe…

I don't think transformers or diffusion models are inherently "unsupervised", especially not the way they're used in Stable Diffusion and related models (which are very much trained in a supervised fashion). I agree with the rest of your points though.

self-supervised is a better term

Re: Stable Video Diffusion

#242

Post by fragmede »

alberth wrote: post #115
mlboss wrote: post #109

Earlier quoted context omitted.

Stable diffusion open source release and llama release

But what technically allowed for so much progress? There’s been open source AI/ML for 20+ years. Nothing comes close to the massive milestones over the past year.

the Transformers are all you need paper from Google, which may end up being a larger contribution to society than Google search, is foundational.

Emad Mostaque and his investment in stable diffusion, and his decision to release it to the world.

I'm sure there are others, but those are the two that stick out to me.

Re: Stable Video Diffusion

#243

Post by keiferski »

Question for anyone more familiar with this space: are there any high-quality tools which take an image and make it into a short video? For example, an image of a tree becomes a video of a tree swaying in the wind.

I have googled for it but mostly just get low quality web tools.

Re: Stable Video Diffusion

#244

Post by lightedman »

arcticbull wrote: post #231
mikebelanger wrote: post #229

Earlier quoted context omitted.

Yeah I'd imagine that's the best way. Lots of LLMs can generate workable Python code too, so code that jives with Blender's Python API doesn't seem like too much of a leap. The only trick is that there has to be enough Blender Python code to train the LLM on.

Maybe something like OpenSCAD is a good middle ground. Procedural code-like format for specifying 3D objects that can then be converted and imported in Blender.

I tried all the AI stuff that I could on OpenSCAD.

While it generates a lot of code that initially makes sense, when you use the code, you get a jumbled block.

Re: Stable Video Diffusion

#245

Post by zeckalpha »

btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

I see that as a reference to the AI generated Toronto Blue Jays advertisement gone wrong that went viral earlier this year. https://www.blogto.com/sports_play/2023/06/ai-generated-toro...

Re: Stable Video Diffusion

#246

Post by ryukoposting »

mattnewton wrote: post #88
dragonwriter wrote: post #66

Earlier quoted context omitted.

Temporal consistency is improving, but “close to being solved” is very optimistic.

No I think we’re actually close. My source is I’m working on this problem and the incredible progress of our tiny 3 person team at drip.art ( http://api.drip.art ) - we can generate a lot of frames that are consistent, and with interpolation between them, smoothly restyle even long videos. Cross-frame attention works for most cases, it just needs to be scaled up. And that’s just for diffusion focused approaches like…

The demo clips on the site are cool, but when you call it a "solved problem," I'd expect to see panning, rotating, and zooming within a cohesive scene with multiple subjects.

Re: Stable Video Diffusion

#247

Post by hackerlight »

jwoodbridge wrote: post #237
TacticalCoder wrote: post #52

Earlier quoted context omitted.

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

we're working on this if you want to give it a try - dream3d.com

You should put a demo on the landing page

Re: Stable Video Diffusion

#249

Post by jojobas »

psunavy03 wrote: post #84
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> sportsball This is not the flex you think it is. You don't have to like sports, but snarking on people who do doesn't make you intellectual, it just makes you come across as a douchebag, no different than a sports fan making fun of "D&D nerds" or something.

Would you get incensed by "petrolhead", "greenfingers" or "trekkie"? Is that what you choose to be emotional about?

Re: Stable Video Diffusion

#250

Post by Hard_Space »

mattnewton wrote: post #88
dragonwriter wrote: post #66

Earlier quoted context omitted.

Temporal consistency is improving, but “close to being solved” is very optimistic.

No I think we’re actually close. My source is I’m working on this problem and the incredible progress of our tiny 3 person team at drip.art ( http://api.drip.art ) - we can generate a lot of frames that are consistent, and with interpolation between them, smoothly restyle even long videos. Cross-frame attention works for most cases, it just needs to be scaled up. And that’s just for diffusion focused approaches like…

Once a video can show a person twisting round, and their belt buckle is the same at the end as it was at the start of the turn, it's solved. VFX pipelines need consistency. TC is a long, long way from being solved, except by hitching it to 3DMMs and SMPL models (and even then, the results are not fabulous yet).
Post reply on HN