No post body was provided.
Stable Diffusion animation
71–80 of 107 posts
Re: Stable Diffusion animation
#72I just came across this on twitter, every frame appears to be an evolution of the previous frame using img2img paired with a tilt/zoom to create a psychedelic animation. The author claims to have made this with Stable Diffusion, Disco, and Wiggle: https://www.youtube.com/watch?v=Nz_n0qxqoPg I believe Wiggle is used to automate the tilt/zoom between frames.
Thats really great and has come a long way since the beginnings... I had this video animation going on back in the day when VQGAN was still all the rave:
https://www.youtube.com/watch?v=CgDbbg802-8
Its incredible what 6 months only can do
Re: Stable Diffusion animation
#73It's clear that the next frontier is to have 3D-space instead of image space transitions. Language itself is very static and action verbs are not enough to specify scene dynamics. I suppose we would need: A. an enriched version of natural language that refines the dynamic processes that occur in a scene B. a data set of isolated processes labeled in the language described in A. I've had a hard time finding ongoing wo…
For 3D we would probably need something like Blender or similar, because at some point it's just easier to use a 3D software to pinpoint where you want stuff to be, than try to use words. Imagine opening blender and typing > A medium sized classroom, well lit, with two blackboards and many geography posters And the AI just generates all the 3D meshes and places them appropriately. Repeat that for other props or chara…
I think you're right about the possibilities here and I love the idea and have thought similar things myself too, but to me the inclusion of a 3D element should probably be a format, not necessarily locked into any specific app such as Blender. Maybe (Pixar) USD is one possible format that could be used for general 3D interchange for this kind of thing?
Re: Stable Diffusion animation
#74Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns. In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart…
Thanks a lot for your insight, it's great to hear from somebody in the industry. This same industry, is logically, also quite keen on enforcing the rights of creators and media copyright.
Most of the discussion, has been on the technical details of these very interesting advances.
Do you think it will be concern for the production process, the origin of the source data used to train these models?
Re: Stable Diffusion animation
#75Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns. In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart…
This sounds like exactly what I always wanted to learn to do, but never knew it existed as a field enough to actually get into it. Honestly thank you so much for this comment.
Do you have resources about getting into this style of art as a novice?
Also what StableDiffusion and other AI tools do you recommend?
Re: Stable Diffusion animation
#76Earlier quoted context omitted.
Maybe GP meant 3080.
For anyone interested in a GPU btw, the 3090 TI had a huge price cut and costs only a bit more than the 3090 right now.
Yup! I've never splurged on a GPU before, but a 3090 TI lets you do textual inversion, and fine tune GPT-J (neither of which you can do with <24gb vram), so now I've got one sitting on my study floor waiting to be installed :)
Re: Stable Diffusion animation
#77Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns. In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart…
I wondered if we were suffering from a collective blind spot where, for example, an outsider looking at copilot might decide that it's revolutionary for writing code where, in practice, I don't know anyone using it in anger.
On the other hand artists are amazingly adept at leveraging new technology, mediums and techniques to create art so it's probably less surprising if they jump at Stable Diffusion in a lot of corners of the industry.
Re: Stable Diffusion animation
#78Earlier quoted context omitted.
You're right, but projects like this do make me think AI handling inbetweens well enough to replace animators is where we'll end up eventually and that it's probably not far off.
I don't want to burst your bubble, but it might happen anyway. The artwork is inside a scene, SD does not understand that scene. The artwork has spatial and human readable emotional relationships. SD does not understand those relationships. SD can maybe create morphs between frames, as a lateral move between two pieces of generated information, but it will never know how to connect up those images in a manner that sa…
> but it will never know how to connect up those images in a manner that satisfies the human requirement of creating a good image.
A few years ago I heard people saying the same thing about going from a piece of text to a picture that "satisfies the human requirement of creating a good image"
Inbetweening is not going to be the obstacle that these AI approaches are finally going to be unable to manage.
Re: Stable Diffusion animation
#79[deleted]
Re: Stable Diffusion animation
#80Earlier quoted context omitted.
I don't want to burst your bubble, but it might happen anyway. The artwork is inside a scene, SD does not understand that scene. The artwork has spatial and human readable emotional relationships. SD does not understand those relationships. SD can maybe create morphs between frames, as a lateral move between two pieces of generated information, but it will never know how to connect up those images in a manner that sa…
> but it will never know how to connect up those images in a manner that satisfies the human requirement of creating a good image. A few years ago I heard people saying the same thing about going from a piece of text to a picture that "satisfies the human requirement of creating a good image" Inbetweening is not going to be the obstacle that these AI approaches are finally going to be unable to manage.
If you say so Captain.