Live data from Hacker News

Stable Diffusion animation

replicate.com

31–40 of 107 posts

Re: Stable Diffusion animation

#31
post #16
post #8

Earlier quoted context omitted.

Neat. Didnt Microsoft release a tool that morphed between two photos a decade ago? I dont recall the name unfortunately, but the effect was similar, less quality.

Back in the DOS days I had (and a still do) a copy of a tool called 'Morph', it took two GIF images, and you placed marker points on the first image and then again on the second, and it generated an MPEG2 file of the morphing. Very impressive for the time.

Mid 90s, baby!

Re: Stable Diffusion animation

#32
post #21

It's clear that the next frontier is to have 3D-space instead of image space transitions. Language itself is very static and action verbs are not enough to specify scene dynamics. I suppose we would need: A. an enriched version of natural language that refines the dynamic processes that occur in a scene B. a data set of isolated processes labeled in the language described in A. I've had a hard time finding ongoing wo…

For 3D we would probably need something like Blender or similar, because at some point it's just easier to use a 3D software to pinpoint where you want stuff to be, than try to use words.

Imagine opening blender and typing

> A medium sized classroom, well lit, with two blackboards and many geography posters

And the AI just generates all the 3D meshes and places them appropriately.

Repeat that for other props or characters that you need. After that you can manually tweak the scene as you currently would (moving things, etc).

Then you select a character, and to animate you tell it

> The character calmly walks to the door and proceeds to open it

You could literally do a 100+ hour job in 5 minutes.

Re: Stable Diffusion animation

#34
post #13

Earlier quoted context omitted.

You can probably expect them for any interesting technology forever into the future, since people have made these useless complaints for hundreds of years at least.

First they came for the horses…. And so on.

Saddle makers had more than six days to prepare!

Re: Stable Diffusion animation

#35
post #13

Earlier quoted context omitted.

You can probably expect them for any interesting technology forever into the future, since people have made these useless complaints for hundreds of years at least.

First they came for the horses…. And so on.

They really did come for the (human) computers, but we got way more jobs out of it than were lost.

Re: Stable Diffusion animation

#36

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

this is very true

Re: Stable Diffusion animation

#37
I played a bit with smoothly interpolating between Stable Diffusion prompts and the effect can be pretty cool but it's hard to avoid discontinuities (like the object changing its orientation), even when using some additional tricks like reusing the previous frame as the initial image or generating several new frames and choosing the one that's closest to the previous frame. You basically have to get lucky with the seed. It probably makes most sense to just wait for video models that take temporal consistency into account explicitly or generate 3D models. There is a lot of promising research out there already, so it's just a matter of time.

Re: Stable Diffusion animation

#39

Last year, when 3090 GPUs were astronomically priced, I thought "screw it, I'll just buy an RTX A5000 for a couple of hundred bucks more." Which begat a second A5000 for "reasons." It was almost prescient. Now all these models are coming out requiring slightly higher VRAM GPUs than a 3090, i.e. more in the range of the A5000, and I get to run them. I am a kid in a candy store this past couple of weeks.

The A5000 and 3090 both have 24GB of ram?

Re: Stable Diffusion animation

#40

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

This is not an example of AI handling the inbetweens. This water-morph effect is undesirable for inbetweens.

You're right, but projects like this do make me think AI handling inbetweens well enough to replace animators is where we'll end up eventually and that it's probably not far off.
Post reply on HN