Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Video Diffusion

stability.ai

Post reply on HN
Print view View original

231–240 of 316 posts

Re: Stable Video Diffusion

#231

Post by arcticbull »

mikebelanger wrote: post #229
mikepurvis wrote: post #118

Earlier quoted context omitted.

But surely you wouldn't try to emit that format directly, but rather some higher level scene description? Or even just a set of instructions for how to manipulate the UI to create the imagined scene?

Yeah I'd imagine that's the best way. Lots of LLMs can generate workable Python code too, so code that jives with Blender's Python API doesn't seem like too much of a leap. The only trick is that there has to be enough Blender Python code to train the LLM on.

Maybe something like OpenSCAD is a good middle ground. Procedural code-like format for specifying 3D objects that can then be converted and imported in Blender.

Re: Stable Video Diffusion

#232

Post by Veraslane »

btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

[dead]

Re: Stable Video Diffusion

#233

Post by Veraslane »

TacticalCoder wrote: post #52
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

[dead]

Re: Stable Video Diffusion

#234

Post by huytersd »

xvector wrote: post #222
ricardobeat wrote: post #221

Earlier quoted context omitted.

> I’ve been advocating for a solid LiDAR camera on the iPhone What do you mean by “advocating”? The iPhone has had a LiDAR camera since 2020.

That's probably why they qualified with "solid", the iPhone's LiDAR camera is quite terrible.

Yes, exactly.

Re: Stable Video Diffusion

#235

Post by numpad0 »

mikepurvis wrote: post #118
boppo1 wrote: post #111

Earlier quoted context omitted.

Blender files are dramatically more complex than any image format, which are basically all just 2D arrays of 3-value vectors. The blender filetype uses a weird DNA/RNA struct system that would probably require its own training run. More on the Blender file format: https://fossies.org/linux/blender/doc/blender_file_format/my...

But surely you wouldn't try to emit that format directly, but rather some higher level scene description? Or even just a set of instructions for how to manipulate the UI to create the imagined scene?

It sure feels weird to me as well, that GenAI is always supposed to be end-to-end with everything done inside NN blackbox. No one seems to be doing image output as SVG or .ai.

Re: Stable Video Diffusion

#236

Post by jwoodbridge »

firefoxd wrote: post #209

I understand the magnitude of innovation that's going on here. But still feel like we are generating these videos with both hands tied behind our backs. In other words, it's nearly impossible to edit the videos in this constraints. (Imagine trying to edit the blue Jays to get the perfect view). Since videos are rarely consumed raw, what if this becomes a pipeline in Blender instead? (Blender the 3d software). Now the…

we're working on this - dream3d.com

Re: Stable Video Diffusion

#237

Post by jwoodbridge »

TacticalCoder wrote: post #52
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

we're working on this if you want to give it a try - dream3d.com

Re: Stable Video Diffusion

#238

Post by COAGULOPATH »

btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> they simultaneously feel crippled and limited by their lack of editing / iteration ability.

Yeah. They're not "videos" so much as images that move around a bit.

This doesn't really look any better than those Midjourney + RunwayML videos we had half a year ago.

>Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close.

Google has a model called Phenaki that supposedly allows for that kind of stuff. But the public can't use it so it's hard to say how good it actually is.

Re: Stable Video Diffusion

#240

Post by COAGULOPATH »

coldtea wrote: post #149
TacticalCoder wrote: post #52

Earlier quoted context omitted.

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

> For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer can immediately spot that. The question is whether the 99% of the audience would even care...

Of course they would. The internet spent a solid month laughing at the Sonic the Hedgehog movie because Sonic had weird-looking teeth.
Post reply on HN