Live data from Hacker News

Stable Diffusion animation

replicate.com

51–60 of 107 posts

Re: Stable Diffusion animation

#51

Earlier quoted context omitted.

First they came for the horses…. And so on.

Well, today they're definitely coming for the artists I've stopped working with for side gigs since dall-e provides good enough results at zero cost and a fraction of the time necessary

>dall-e provides good enough results at zero cost

Even the direct costs of using OpenAI APIs are not zero.

Re: Stable Diffusion animation

#52
Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns.

In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart other images and stitch them together to get the desired image. Some start with a rough sketch for the composition or make really rough grey shade 3d model and "overpaint" them. When it comes to "photobashing" the disregard for copyrights was always there and it's the worst in smaller studios and a bit better in the leading ones. Still most of the time everyone argues that if you only use really small parts of the images it is covered by fair use. There are some examples were studios got sued but mostly without bigger financial impact.

A few months ago I started working with "DiscoDiffusion" to generate the images I use to photobash. "DiscoDiffusion" can produce great "painterly" images but struggles with photorealism and is slower, not as coherent as "StableDiffusion". Still the adoption rate in the concept art community was insanely fast. This all got topped by "StableDiffusion" in the last week. Ofc there are still people that want to do it the "right" way and not use AI but we had the same discussion years ago when "photobashing" came into place and some artists still wanted to paint the whole image. As concept artist you are mostly paid for your design thinking that means it is less about the process and more about the finished product. The turnaround time for styleframes got reduced from 3-4 hours while painting to 45 min - 1 hour when photobashing with stable diffusion me and my peers in the studio are now at 20-45 min per styleframe. When "photobashing" most people constrain themselves on their image library and ressources like Photobashing Kits. Not only does "StableDiffusion" cut the time in half it also gives greater freedom in composition and design especially if you are using img2img.

So where does this leave us? For the work in fast paced art environments like VFX, games, conept art or advertising "StableDiffusion" is a welcome gamechanger. Tradionalists and Artists outside of the industry might feel threatened but for us in these industries it's a god send.

Re: Stable Diffusion animation

#53

Earlier quoted context omitted.

Well, today they're definitely coming for the artists I've stopped working with for side gigs since dall-e provides good enough results at zero cost and a fraction of the time necessary

>dall-e provides good enough results at zero cost Even the direct costs of using OpenAI APIs are not zero.

for my needs I didn't have to shell out a single € yet

Re: Stable Diffusion animation

#55
post #50

Very cool! I generated this with 1000 images https://twitter.com/UnshushProject/status/156315821457709465... using the Deforum's Colab[1], it's really easy and now has interpolation too. It was the very first video, I could have made something great but, you know, awesome guys keep releasing AI tech and I'm like a child at Luna Park right now, not able to concentrate. If you are interested in my project (I doubt, you…

I'm definitely interested in your video related bookmarks. You have generated some pretty cool designs.

Sure! This would be my approach (and tools) if I was smarter:

If you make the generations with some similarities and use the right interpolation, you don't need 1000 images like my video and can obtain a smooth movement.

First, generate images with some kind of visual anchor (background, an object). You can use frames generated using the previous frame as reference image, or the same seed but different prompt/parameters, or you can go wild using img2img/inpainting (btw I struggle to find an inpainting tool for Stable Diffusion: they seem to be just img2img with a mask, without contest).

Then pass the generated images to one of the most recent interpolation algorithms, like this one https://github.com/megvii-research/ECCV2022-RIFE or the one used in the replicate we are commenting on (someone posted this reference: https://github.com/google-research/frame-interpolation )

The first link reports some free and paid implementation and a Colab, so depending on how deep you want to go, you have a lot of choices.

In the end, I'd use some good app to stabilize the image if needed, to get a more "calm" look. I use Luma Fusion, but it's a paid app (cheap, one-time payment, for iOS). I'm sure there are a ton of open-source implementations.

It's an approach similar to the animation on replicate, but it allows a lot of fine-tuning and you can add new animation ideas/tools to the process.

Nothing revolutionary, but I hope it helps!

> You have generated some pretty cool designs.

Thanks! I put in a lot of work in the last weeks. The project has a mission, I wrote something, but it's not ready yet. I believe it will be with the launch of Dall-E 8 :-/

Re: Stable Diffusion animation

#57

Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns. In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart…

I think Tradionalists and Artists do what they do because they enjoy the process of creating/creating something unique, not because they want to be productive/fast.

So yeah, StableDif is great for commercially constrained environments, plus, it frees some of your time so you can enjoy creating art that matters to you :p

Re: Stable Diffusion animation

#58

Maybe I can shine some light on the debate from an concept artist standpoint that works in VFX and advertising. I worked on feature films (3 of them in the imdb top 100), tv shows (like game of thrones) and hundreds of AD campaigns. In the last 10 year the work of a concept artist changed dramatically we have gone from purely painted concept art to mostly "photobashed". Photobashed means basically that you rip apart…

Awesome perspective. Would you be able to link to an an example of what a finished style frame looks like using Stable Diffusion? I am curious what human and machine can achieve in 20 minutes.

Re: Stable Diffusion animation

#59
My predictions for 10 - 15 years:

- Mandela effect for famous art pieces: "Monalisa was AI generated" "No it wasn't" "Yes it was".

- Art critics will get the last laugh, as people start giving them truckloads of money to ask whether a piece of art is human or AI generated.

Re: Stable Diffusion animation

#60
post #39

Earlier quoted context omitted.

The A5000 and 3090 both have 24GB of ram?

I should have mentioned linking the GPUs. I meant requiring slightly more VRAM than a single 3090 can handle. Or a 3080 as others have pointed out. The main difference between the way the 3090 works and the A5000 works is SLI vs NvLink/NvSwitch. I believe the 3090 uses NvLink, but not quite in the same way the A5000 does. I can chain together far more A5000's than I can 3090's. Eight A500's vs four 3090's IIRC. And e…

What are the specs of your setup? How high is the power draw?
Post reply on HN