71–80 of 184 posts
Great stuff. I like that they give directions for both Swift and Python This gets you text descriptions to images. I have seen models that given a picture, then generate similar pictures. I want this because while I have many pictures of my grandmothers, I only have a couple of pictures of my grandfathers and it would be nice to generate a few more. Core ML is so well done. A year ago I wrote a book on Swift AI and u…
That’s DreamBooth. There are some services that will do it for you.
How come you always have to install some version of pytorch or tensor flow to run these ml models? When I'm only doing inference shouldn't there be easier ways of doing that, with automatic hardware selection etc. Why aren't models distributed in a standard format like onnx, and inference on different platforms solved once per platform?
Apple has their own mlmodel format but they can’t distribute this model as a direct download due to the models EULA. The first task is to translate the model.
How does this compare with using the Hugging Face `diffusers` package with MPS acceleration through PyTorch Nightly? I was under the impression that that used CoreML under the hood as well to convert the models so they ran on the Neural Engine.
It doesn't. MPS largely is on GPU. PyTorch's MPS implementation is incomplete a few weeks ago as well. This is about 3x faster.
Earlier quoted context omitted.
i'm very ignorant here so forgive me but if it can generate images that fast can it be used to generate a video?
Video is really a series of frames, the framerate for film/human can get away with 24 frames/second-- so maybe ~40ms/image for real-time at least? What's cool about the era in which we live is if you look at high-performance graphics for games or simulations, for instance, it may in fact be faster to a the model to "enhance" a low-resolution frame rather than trying to render it fully on the machine. ex. AMD's FSR vs…
Atila from Apple on the expected performance: > For distilled StableDiffusion 2 which requires 1 to 4 iterations instead of 50, the same M2 device should generate an image in https://twitter.com/atiorh/status/1598399408160342039
i'm very ignorant here so forgive me but if it can generate images that fast can it be used to generate a video?
See deforum[1] and andreasjansson‘s stable-diffusion-animation[2]
[1]: https://deforum.github.io/
[2]: https://replicate.com/andreasjansson/stable-diffusion-animat...
Atila from Apple on the expected performance: > For distilled StableDiffusion 2 which requires 1 to 4 iterations instead of 50, the same M2 device should generate an image in https://twitter.com/atiorh/status/1598399408160342039
If you told me this was possible when I bought an M1 Pro less than a year ago, I wouldn’t believe you. This is insane.
And the posted benchmarks for the M2 Macbook Air make me consider 'upgrading' to an Air.
Man, this takes a ton of room to do the CoreML conversions - ran out of space doing the unet conversion even though I started with 25GB free. Going on a delete spree to get it up to 50GB free before trying again.
Edit: still alive! https://grandperspectiv.sourceforge.net/