Live data from Hacker News

Stable Diffusion animation

replicate.com

11–20 of 107 posts

Re: Stable Diffusion animation

#11

I just came across this on twitter, every frame appears to be an evolution of the previous frame using img2img paired with a tilt/zoom to create a psychedelic animation. The author claims to have made this with Stable Diffusion, Disco, and Wiggle: https://www.youtube.com/watch?v=Nz_n0qxqoPg I believe Wiggle is used to automate the tilt/zoom between frames.

Interesting to see in comparison to those old latent space zoom vids making the rounds in like ~2014..

Re: Stable Diffusion animation

#12

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

> leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea!

These comments on every single post are getting really boring.

Re: Stable Diffusion animation

#13

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

> leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! These comments on every single post are getting really boring.

You can probably expect them for any interesting technology forever into the future, since people have made these useless complaints for hundreds of years at least.

Re: Stable Diffusion animation

#14
post #8
post #2

In this example, 25 frames are generated using Stable Diffusion, then frames are interpolated using FILM-Net. I hadn't see FILM-net before, it looks really neat.

Neat. Didnt Microsoft release a tool that morphed between two photos a decade ago? I dont recall the name unfortunately, but the effect was similar, less quality.

I was doing this with an After Effects plugin called Twixtor long before a decade ago.

https://revisionfx.com/products/twixtor/

Re: Stable Diffusion animation

#15

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

This is not an example of AI handling the inbetweens.

This water-morph effect is undesirable for inbetweens.

Re: Stable Diffusion animation

#16
post #8
post #2

In this example, 25 frames are generated using Stable Diffusion, then frames are interpolated using FILM-Net. I hadn't see FILM-net before, it looks really neat.

Neat. Didnt Microsoft release a tool that morphed between two photos a decade ago? I dont recall the name unfortunately, but the effect was similar, less quality.

Back in the DOS days I had (and a still do) a copy of a tool called 'Morph', it took two GIF images, and you placed marker points on the first image and then again on the second, and it generated an MPEG2 file of the morphing.

Very impressive for the time.

Re: Stable Diffusion animation

#17
post #8
post #2

In this example, 25 frames are generated using Stable Diffusion, then frames are interpolated using FILM-Net. I hadn't see FILM-net before, it looks really neat.

Neat. Didnt Microsoft release a tool that morphed between two photos a decade ago? I dont recall the name unfortunately, but the effect was similar, less quality.

You're thinking of Microsoft PhotoSynth from 2006. This is similar to what google streetview showed up with.

From what I can see, they later revamped PhotoSynth to include actual 3d mesh reconstruction in 2014.

Re: Stable Diffusion animation

#18
post #13

Earlier quoted context omitted.

> leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! These comments on every single post are getting really boring.

You can probably expect them for any interesting technology forever into the future, since people have made these useless complaints for hundreds of years at least.

First they came for the horses….

And so on.

Re: Stable Diffusion animation

#19
post #8
post #2

In this example, 25 frames are generated using Stable Diffusion, then frames are interpolated using FILM-Net. I hadn't see FILM-net before, it looks really neat.

Neat. Didnt Microsoft release a tool that morphed between two photos a decade ago? I dont recall the name unfortunately, but the effect was similar, less quality.

Morphing is different to interpolation and has been available on consumer computers since the 1990s (I remember making gifs in 1996 with it).

In morphing you end up with a blury mess for the between frames. This technique tracks individual features (eg eyes) and keeps them coherent.

Re: Stable Diffusion animation

#20

AI in animation has been interesting to me for a while now. It leaves me a little conflicted though. If we get to the point where we can throw key drawings at AI and let it handle all the inbewteens without a bunch of tweaking and cleanup afterwards it's going to really suck for places like Korea! I guess all those inbeatweeners will just be another victim of automation. I've always loved animation, but I'll admit pa…

> It's pure insanity that people ever drew, by hand, mountains of individual drawings each slightly changed and assembled them into compelling illusions to tell stories.

Traditional animation by itself is nothing short of insanity, convincingly blending live action and traditional animation takes it a step further, and then there's the "Bumping the Lamp"[0] scene in Who Framed Roger Rabbit.

The film as a whole refuses to keep the camera static to make things easy for the animators, which was unusual enough by itself, but then they went above and beyond — they casually bumped a pendant lamp and let it flail about. Every time it slows down, it gets bumped again. And they shaded and cast shadows for the damned rabbit for every single frame of that sequence. Madness.

0. https://www.youtube.com/watch?v=_EUPwsD64GI

Post reply on HN