Running Stable Diffusion in 260MB of RAM

#1

Post by Robin89 » Thu, Jul 20, 2023, 5:01 PM UTC

Running Stable Diffusion in 260MB of RAM
github.com

Re: Running Stable Diffusion in 260MB of RAM

#2

Post by boredemployee » Thu, Jul 20, 2023, 6:03 PM UTC

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD.

I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

Re: Running Stable Diffusion in 260MB of RAM

#3

Post by wsgeorge » Thu, Jul 20, 2023, 6:20 PM UTC

boredemployee wrote: post #2

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

On an Apple M1 with 16gig RAM, without using Pytorch compiled to take advantage of Metal, it could take 12mins to generate an image with a tweet-length prompt. With Metal, it takes less than 60 seconds.

Re: Running Stable Diffusion in 260MB of RAM

#4

Post by mottiden » Thu, Jul 20, 2023, 6:22 PM UTC

Amazing work!

Re: Running Stable Diffusion in 260MB of RAM

#5

Post by stmblast » Thu, Jul 20, 2023, 6:29 PM UTC

This is really neat! Always cool to see what people can do with less.

Re: Running Stable Diffusion in 260MB of RAM

#6

Post by atrus » Thu, Jul 20, 2023, 6:29 PM UTC

boredemployee wrote: post #2

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

I was using a 6ish year old amd cpu with 16gigs of ram and generating a prompt would take about a half hour. Which is still massively impressive for what it is.

Re: Running Stable Diffusion in 260MB of RAM

#7

Post by vikasr111 » Thu, Jul 20, 2023, 6:34 PM UTC

Interesting. Which platform/PC config did you use?

Re: Running Stable Diffusion in 260MB of RAM

#8

Post by crest » Thu, Jul 20, 2023, 6:35 PM UTC

Does this mean you could fit its whole working set in the cache hierarchy of a modern high end GPU getting near 100% ALU utilisation?

Re: Running Stable Diffusion in 260MB of RAM

#9

Post by microtonal » Thu, Jul 20, 2023, 6:52 PM UTC

wsgeorge wrote: post #3

boredemployee wrote: post #2

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

On an Apple M1 with 16gig RAM, without using Pytorch compiled to take advantage of Metal, it could take 12mins to generate an image with a tweet-length prompt. With Metal, it takes less than 60 seconds.

And PyTorch on the M1 (without Metal) uses the fast AMX matrix multiplication units (through the Accelerate Framework). The matrix multiplication on the M1 is on par with ~10 threads/cores of Ryzen 5900X.

[1] https://github.com/danieldk/gemm-benchmark#example-results

Re: Running Stable Diffusion in 260MB of RAM

#10

Post by londons_explore » Thu, Jul 20, 2023, 6:53 PM UTC

atrus wrote: post #6

boredemployee wrote: post #2

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

I was using a 6ish year old amd cpu with 16gigs of ram and generating a prompt would take about a half hour. Which is still massively impressive for what it is.

Use a free GPU from google colab and you can do the same in about 15 seconds...