Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion animation

replicate.com

Post reply on HN
Print view View original

21–30 of 107 posts

Re: Stable Diffusion animation

#21

Post by Lwepz »

It's clear that the next frontier is to have 3D-space instead of image space transitions. Language itself is very static and action verbs are not enough to specify scene dynamics. I suppose we would need: A. an enriched version of natural language that refines the dynamic processes that occur in a scene B. a data set of isolated processes labeled in the language described in A.

I've had a hard time finding ongoing work on A. and B, perhaps it isn't much of a priority for research groups.

Re: Stable Diffusion animation

#22

Post by NoMoreBro »

Very cool! I generated this with 1000 images https://twitter.com/UnshushProject/status/156315821457709465... using the Deforum's Colab[1], it's really easy and now has interpolation too. It was the very first video, I could have made something great but, you know, awesome guys keep releasing AI tech and I'm like a child at Luna Park right now, not able to concentrate.

If you are interested in my project (I doubt, you are too busy playing like me) I'm posting a lot of things on https://unshush.com and on the Instagram account: https://www.instagram.com/unshushproject/ (Sorry for posting my stuff but I'm not very social so no one will ever see them)

If you want to generate videos I can share some links I bookmarked of software/code to make them more smooth.

[1] Deforum's Colab (based on Stable Diffusion): https://colab.research.google.com/github/deforum/stable-diff...

Re: Stable Diffusion animation

#23

Post by alx__ »

r3trohack3r wrote: post #7

I just came across this on twitter, every frame appears to be an evolution of the previous frame using img2img paired with a tilt/zoom to create a psychedelic animation. The author claims to have made this with Stable Diffusion, Disco, and Wiggle: https://www.youtube.com/watch?v=Nz_n0qxqoPg I believe Wiggle is used to automate the tilt/zoom between frames.

That is wild, feels like an intense dream you can't wake up from

Re: Stable Diffusion animation

#25

Post by jcelerier »

quickthrower2 wrote: post #18
vasco wrote: post #13

Earlier quoted context omitted.

You can probably expect them for any interesting technology forever into the future, since people have made these useless complaints for hundreds of years at least.

First they came for the horses…. And so on.

Well, today they're definitely coming for the artists I've stopped working with for side gigs since dall-e provides good enough results at zero cost and a fraction of the time necessary

Re: Stable Diffusion animation

#26

Post by justinlloyd »

Last year, when 3090 GPUs were astronomically priced, I thought "screw it, I'll just buy an RTX A5000 for a couple of hundred bucks more." Which begat a second A5000 for "reasons." It was almost prescient. Now all these models are coming out requiring slightly higher VRAM GPUs than a 3090, i.e. more in the range of the A5000, and I get to run them. I am a kid in a candy store this past couple of weeks.

Re: Stable Diffusion animation

#28

Post by actionfromafar »

r3trohack3r wrote: post #7

I just came across this on twitter, every frame appears to be an evolution of the previous frame using img2img paired with a tilt/zoom to create a psychedelic animation. The author claims to have made this with Stable Diffusion, Disco, and Wiggle: https://www.youtube.com/watch?v=Nz_n0qxqoPg I believe Wiggle is used to automate the tilt/zoom between frames.

I like their earlier work more, with the audio pictograms.

Re: Stable Diffusion animation

#29

Post by fagerhult »

Andreas, author of the Replicate model here -- though "author" feels wrong since I basically just stitched two amazing models together.

The thing that really strikes me is that open source ML is starting to behave like open source software. I was able to take a pretrained text-to-image model and combine it with a pretrained video frame interpolation model and the two actually fit together! I didn't have to re-train or fine tune or map between incompatible embedding spaces, because these models can generalize to basically any image. I could treat these models as modular building blocks.

It just makes your creative mind spin. What if you generate some speech with https://replicate.com/afiaka87/tortoise-tts, generate an image of an alien with Stable Diffusion, and then feed those two into https://replicate.com/wyhsirius/lia. Talking alien! Machine learning is starting to become really fun, even if you don't know anything about partial derivatives.

Re: Stable Diffusion animation

#30

Post by kristopolous »

zmmmmm wrote: post #6
mrpf1ster wrote: post #3

I feel like I’m watching an explosion of progress in AI image generation in real-time. Every day there’s a new application of Stable Diffusion. It’s incredible to watch unfold

I think it's pretty notable how most of the explosion happened since stable diffusion released their model and code as open source, while Dall-E generated initial excitement with their closed source model, but limited progress / creativity since. It's a pretty nice demonstration I think of how much innovation can happen from openness.

Yes. Example #5*10^7 or so. Some people are just opposed ideologically or due to temperament. Locking things down is one of the best ways to make them die
Post reply on HN