Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Video Diffusion

stability.ai

Post reply on HN
Print view View original

131–140 of 316 posts

Re: Stable Video Diffusion

#131

Post by KineticLensman »

freedomben wrote: post #93
KineticLensman wrote: post #68

Earlier quoted context omitted.

(disclaimer: worked in the sim industry for 25 years, still active in terms of physics-based rendering). First off, there are zero technical proofs that we are in a sim, just a number of philosophical arguments. In practical terms, we cannot yet simulate a single human cell at the molecular level, given the massive number of interactions that occur every microsecond. Simulating our entire universe is not technically…

I don't think you would need to simulate the entire universe, just enough of it that the consciousness receiving sense data can't encounter any missing info or "glitches" in the metaphorical matrix. Still hard of course, but substantially less compute intensive than every molecule in the universe.

> but substantially less compute intensive than every molecule in the universe

Very true, but to me this view of the universe and one's existence within it as a sort of second-rate solipsist bodge isn't a satisfyingly profound answer to the question of life the universe and everything.

Although put like that it explains quite a lot.

[Edit] There is also a sense in which the sim-as-a-focussed-mini-universe view is even less falsifiable, because sim proponents address any doubt about the sim by moving the goal posts to accommodate what they claim is actually achievable by the putative creator/hacker on Planet Tharg or similar.

Re: Stable Video Diffusion

#132

Post by mkaic »

speedgoose wrote: post #86

Has anyone managed to run the thing? I got the streamlit demo to start after fighting with pytorch, mamba, and pip for half an hour, but the demo runs out of GPU memory after a little while. I have 24GB on GPU on the machine I used, does it need more?

Have heard from others attempting it that it needs 40GB, so basically an A100/A6000/H100 or other large card. Or an Apple Silicon Mac with a bunch of unified memory, I guess.

Re: Stable Video Diffusion

#133

Post by bozhark »

epr wrote: post #104
TacticalCoder wrote: post #52

Earlier quoted context omitted.

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

> you'll describe something and you'll get a full 3D scene, with 3D models, source of lights set up, etc. I'm always confused why I don't hear more about projects going in this direction. Controlnets are great, but there's still quite a lot of hallucination and other tiny mistakes that a skilled human would never make.

One was on the front page the other day, I’ll search for a link

Re: Stable Video Diffusion

#134

Post by smcleod »

xianshou wrote: post #33
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

Emu edit should be exactly what you're looking for: https://ai.meta.com/blog/emu-text-to-video-generation-image-...

It doesn’t look like the code for that is available anywhere though?

Re: Stable Video Diffusion

#135

Post by dragonwriter »

kmeisthax wrote: post #124
ericpauley wrote: post #8

I'm still puzzled as to how these "non-commercial" model licenses are supposed to be enforceable. Software licenses govern the redistribution of the software , not products produced with it. An image isn't GPL'd because it was produced with GIMP.

So, there's a few different things interacting here that are a little confusing. First off, you have copyright law, which grants monopolies on the act of copying to the creators of the original. In order to legally make use of that work you need to either have permission to do so (a license), or you need to own a copy of the work that was made by someone with permission to make and sell copies (a sale). For the purpo…

> Licenses are, at least under US law, contracts

Not necessarily; gratuitous licenses are not contracts. Licenses which happen to also meet the requirements for contracts (or be embedded in agreements that do) are contracts or components of contracts, but that's not all licenses.

Re: Stable Video Diffusion

#136

Post by kmeisthax »

alberth wrote: post #115
mlboss wrote: post #109

Earlier quoted context omitted.

Stable diffusion open source release and llama release

But what technically allowed for so much progress? There’s been open source AI/ML for 20+ years. Nothing comes close to the massive milestones over the past year.

Attention, transformers, diffusion. Prior image synthesis techniques - i.e. GANs - had problems that made it difficult to scale them up, whereas the current techniques seem to have no limit other than the amount of RAM in your GPU.

Re: Stable Video Diffusion

#137

Post by mlboss »

mkaic wrote: post #132
speedgoose wrote: post #86

Has anyone managed to run the thing? I got the streamlit demo to start after fighting with pytorch, mamba, and pip for half an hour, but the demo runs out of GPU memory after a little while. I have 24GB on GPU on the machine I used, does it need more?

Have heard from others attempting it that it needs 40GB, so basically an A100/A6000/H100 or other large card. Or an Apple Silicon Mac with a bunch of unified memory, I guess.

Give it a week.

Re: Stable Video Diffusion

#138

Post by speedgoose »

mkaic wrote: post #132
speedgoose wrote: post #86

Has anyone managed to run the thing? I got the streamlit demo to start after fighting with pytorch, mamba, and pip for half an hour, but the demo runs out of GPU memory after a little while. I have 24GB on GPU on the machine I used, does it need more?

Have heard from others attempting it that it needs 40GB, so basically an A100/A6000/H100 or other large card. Or an Apple Silicon Mac with a bunch of unified memory, I guess.

Alright thanks for the information. I will try to justify using one A100 for my "very important" research activities.

Re: Stable Video Diffusion

#139

Post by BirdieNZ »

mikepurvis wrote: post #118
boppo1 wrote: post #111

Earlier quoted context omitted.

Blender files are dramatically more complex than any image format, which are basically all just 2D arrays of 3-value vectors. The blender filetype uses a weird DNA/RNA struct system that would probably require its own training run. More on the Blender file format: https://fossies.org/linux/blender/doc/blender_file_format/my...

But surely you wouldn't try to emit that format directly, but rather some higher level scene description? Or even just a set of instructions for how to manipulate the UI to create the imagined scene?

I've seen this but producing Python scripts that you run in Blender, e.g. https://www.youtube.com/watch?v=x60zHw_z4NM (but I saw something marginally more impressive, not sure where though!)

Re: Stable Video Diffusion

#140

Post by internet101010 »

TacticalCoder wrote: post #52
btbuildem wrote: post #11

In the video towards the bottom of the page, there are two birds (blue jays), but in the background there are two identical buildings (which look a lot like the CN Tower). CN Tower is the main landmark of Toronto, whose baseball team happens to be the Blue Jays. It's located near the main sportsball stadium downtown. I vaguely understand how text-to-image works, and so it makes sense that the vector space for "blue j…

> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…

I am guessing it will be similar to inpainting in normal stable diffusion, which is easy when using the workflow feature InvokeAI ui.
Post reply on HN