Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Video Diffusion

stability.ai

Post reply on HN
Print view View original

221–230 of 316 posts

Re: Stable Video Diffusion

#221

Post by ricardobeat »

huytersd wrote: post #214
firefoxd wrote: post #209

I understand the magnitude of innovation that's going on here. But still feel like we are generating these videos with both hands tied behind our backs. In other words, it's nearly impossible to edit the videos in this constraints. (Imagine trying to edit the blue Jays to get the perfect view). Since videos are rarely consumed raw, what if this becomes a pipeline in Blender instead? (Blender the 3d software). Now the…

What would solve all these issues is full generation of 3D models that we hopefully get a chance to see over the next decade. I’ve been advocating for a solid LiDAR camera on the iPhone so there is a lot of training data for these LLMs.

> I’ve been advocating for a solid LiDAR camera on the iPhone

What do you mean by “advocating”? The iPhone has had a LiDAR camera since 2020.

Re: Stable Video Diffusion

#222

Post by xvector »

ricardobeat wrote: post #221
huytersd wrote: post #214

Earlier quoted context omitted.

What would solve all these issues is full generation of 3D models that we hopefully get a chance to see over the next decade. I’ve been advocating for a solid LiDAR camera on the iPhone so there is a lot of training data for these LLMs.

> I’ve been advocating for a solid LiDAR camera on the iPhone What do you mean by “advocating”? The iPhone has had a LiDAR camera since 2020.

That's probably why they qualified with "solid", the iPhone's LiDAR camera is quite terrible.

Re: Stable Video Diffusion

#223

Post by eigenvalue »

epr wrote: post #104
TacticalCoder wrote: post #52

Earlier quoted context omitted.

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

> you'll describe something and you'll get a full 3D scene, with 3D models, source of lights set up, etc. I'm always confused why I don't hear more about projects going in this direction. Controlnets are great, but there's still quite a lot of hallucination and other tiny mistakes that a skilled human would never make.

I think this recent Gaussian Splatting technique could end up working really well for generative models, at least once there is a big corpus of high quality scenes to train on. Seems almost ideal for the task because it gets photorealistic results from any angle, but in a sparse, data efficient way, and it doesn’t require a separate rendering pipeline.

Re: Stable Video Diffusion

#225

Post by dsmmcken »

btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

Adobe is doing some great work here in my opinion in terms of building AI tools that make sense for artist workflows. This "sneak peak" demo from the recent Adobe Max conference is pretty much exactly what you described, actually better because you can just click on an object in the image and drag it.

See video: https://www.adobe.com/max/2023/sessions/project-stardust-gs6...

Re: Stable Video Diffusion

#226

Post by frognumber »

yorwba wrote: post #184
ronsor wrote: post #153

Earlier quoted context omitted.

> Then the creator can still require people to agree to contractual terms before downloading the model from them, presumably including the usage limitations as well as an agreement not to redistribute the model to anyone else who does not also agree. I don't think it's possible to invent copyright-like rights.

Why not? Two willing parties can agree to bind themselves to all kinds of obligations in a contract as long as they're not explicitly illegal. Copyleft is an example of someone successfully inventing a copyright-like right by bootstrapping off existing copyright with a specially engineered contract.

There are a few problems:

1) You and I invent our own private "copyright" for data (which is not copyrightable)

2) Everything is fine until my wife walks up to my computer and makes a copy of the data. She's not bound by our private "copyright." She doesn't even know it exists, and shares the data with her bestie.

And... our private pseudo-copyright is dead.

Also: Licenses are not the same as contracts. There are times when something can be both, one, or the other. But there are a lot of limits on how far they reach. The output of a program is rarely copyrightable by the author (as opposed to the user).

Re: Stable Video Diffusion

#227

Post by rbhuta »

gregorymichael wrote: post #220

How long until Replicate has this available?

We're hosting this free (no credit card needed) at https://app.decoherence.co/stablevideo Disclaimer: Google log-in required to help us reduce spam.

Let me know what you think of it! It works best on landscape images from my tests.

Re: Stable Video Diffusion

#228

Post by rbhuta »

nuclearsugar wrote: post #190

Very excited to play with this. Some of my latest experiments - https://www.jasonfletcher.info/vjloops/

We're hosting this free (no credit card needed) at https://app.decoherence.co/stablevideo Disclaimer: Google log-in required to help us reduce spam. Let me know what you think of it! It works best on landscape images from my tests.

Re: Stable Video Diffusion

#229

Post by mikebelanger »

mikepurvis wrote: post #118
boppo1 wrote: post #111

Earlier quoted context omitted.

Blender files are dramatically more complex than any image format, which are basically all just 2D arrays of 3-value vectors. The blender filetype uses a weird DNA/RNA struct system that would probably require its own training run. More on the Blender file format: https://fossies.org/linux/blender/doc/blender_file_format/my...

But surely you wouldn't try to emit that format directly, but rather some higher level scene description? Or even just a set of instructions for how to manipulate the UI to create the imagined scene?

Yeah I'd imagine that's the best way. Lots of LLMs can generate workable Python code too, so code that jives with Blender's Python API doesn't seem like too much of a leap.

The only trick is that there has to be enough Blender Python code to train the LLM on.

Re: Stable Video Diffusion

#230

Post by xena »

rbhuta wrote: post #224

VRAM requirements are big for this launch. We're hosting this for free at https://app.decoherence.co/stablevideo . Disclaimer: Google log-in required to help us reduce spam.

How big is big?
Post reply on HN