21–30 of 66 posts
Earlier quoted context omitted.
Make sure you build in a capacity to save all the previous iterations in case you see something you really like.
Haha I like the idea of walking past, glancing now and then to see if there's something you really love... but on the other hand I would also love the statement behind something unconnected to the internet that's slowly churning out unique, ephemeral pictures. Yours to enjoy, then gone forever.
Incredible! The march continues to get more models to run on the edge, much faster than I anticipated. The static quantization and slicing techniques here are pretty cool
Earlier quoted context omitted.
Prompt length shouldn't influence creation time, at least it didn't in any of the implementations I used. What is the resolution of your images and number of steps?
Defaults from the Huggingface repo, just copy-pasted. So, iirc 50 steps and the image is 512x512. Edit: confirmed. > Prompt length shouldn't influence creation time... Yeah, checks out with my experience too. Longer prompts were truncated.
Albeit in 77 token increments.
That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?
On an Apple M1 with 16gig RAM, without using Pytorch compiled to take advantage of Metal, it could take 12mins to generate an image with a tweet-length prompt. With Metal, it takes less than 60 seconds.
Wait are these inference times real? 1 second on a Raspi? Do I get this right? This is faster than on my GPU. What's going on here?
update: "Tests were run on my development machine: Windows Server 2019, 16GB RAM, 8750H cpu (AVX2), 970 EVO Plus SSD, 8 virtual cores on VMWare."
Wait are these inference times real? 1 second on a Raspi? Do I get this right? This is faster than on my GPU. What's going on here?
Check back in a few months for my results...
Does this mean you could fit its whole working set in the cache hierarchy of a modern high end GPU getting near 100% ALU utilisation?