Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion in 260MB of RAM

github.com

Post reply on HN
Print view View original

41–50 of 66 posts

Re: Running Stable Diffusion in 260MB of RAM

#41

Post by LordDragonfang »

Minor49er wrote: post #38
zirgs wrote: post #30

"It runs Stable Diffusion" is the new "It runs Doom".

Now I'm wondering: could a monkey hitting random keys on a keyboard for an infinite amount of time eventually come up with the right prompts to get GPT-4 to produce code that compiles to a faithful reproduction of Doom?

Probably more easily than you'd think. DOOM is open source[1], and as GP alludes, is probably the most frequently ported game in existence, so its source code almost certainly appears multiple times in GPT-4's training set, likely alongside multiple annotated explanations.

[1] https://github.com/id-Software/DOOM

Re: Running Stable Diffusion in 260MB of RAM

#42

Post by hadlock »

speedgoose wrote: post #36
hadlock wrote: post #35

Earlier quoted context omitted.

For on prem use, the up front cost is a lot lower. The A100 that most serious outfits are using runs in the thousands to tens of thousands of dollars per unit with very limited availability. The pi is typically under $75 usd for any variant.

A RTX 4090 has a much better value for stable diffusion but yes if you start to think about cost the pi wins. If you think about availability, I’m not sure.

The big immediate plus here, is if you live somewhere with limited access to the internet, you can still generate imagery offline on a low end laptop, like a protest group in far eastern europe or other areas. My personal travel laptop only has 8GB memory so it's exciting to be able to try out an idea even if I don't have high end hardware.

Re: Running Stable Diffusion in 260MB of RAM

#43

Post by [deleted] »

LordDragonfang wrote: post #41
Minor49er wrote: post #38

Earlier quoted context omitted.

Now I'm wondering: could a monkey hitting random keys on a keyboard for an infinite amount of time eventually come up with the right prompts to get GPT-4 to produce code that compiles to a faithful reproduction of Doom?

Probably more easily than you'd think. DOOM is open source[1], and as GP alludes, is probably the most frequently ported game in existence, so its source code almost certainly appears multiple times in GPT-4's training set, likely alongside multiple annotated explanations. [1] https://github.com/id-Software/DOOM

[deleted]

Re: Running Stable Diffusion in 260MB of RAM

#44

Post by kuchenbecker »

asynchronous wrote: post #22
saqadri wrote: post #14

Incredible! The march continues to get more models to run on the edge, much faster than I anticipated. The static quantization and slicing techniques here are pretty cool

I’ve been amazed at how quickly the open source community has iterated on LLMs and Diffusion models. Goes to show how well open source can work.

Innovation in the tech world is spurred by open access.

Re: Running Stable Diffusion in 260MB of RAM

#45

Post by anthk »

LordDragonfang wrote: post #41
Minor49er wrote: post #38

Earlier quoted context omitted.

Now I'm wondering: could a monkey hitting random keys on a keyboard for an infinite amount of time eventually come up with the right prompts to get GPT-4 to produce code that compiles to a faithful reproduction of Doom?

Probably more easily than you'd think. DOOM is open source[1], and as GP alludes, is probably the most frequently ported game in existence, so its source code almost certainly appears multiple times in GPT-4's training set, likely alongside multiple annotated explanations. [1] https://github.com/id-Software/DOOM

Well, not the most ported, the Z-Machine with tons of games (even ones legally available from IF archive with great programming, such as Curses!, Jigsaw, Anchorhead) might be. It runs even on the Game Boy, up to v3 games. Z5 and Z8 games will run fine from a 68020 and beyond.

Re: Running Stable Diffusion in 260MB of RAM

#46

Post by xtagon »

isoprophlex wrote: post #12

Incredible! If only there was some cheap hackable eink frame, you could make a fully self contained artwork from eink panel + rpi that's (slowly) continuously updating itself..!

Waveshare and Pimoroni have some that work well with Raspberry Pi, if they're in your budget. I built a Waveshare epaper display + Pi Zero into a photo frame for a totally different project. Your idea tempts me.

Re: Running Stable Diffusion in 260MB of RAM

#47

Post by hospitalJail »

kuchenbecker wrote: post #44
asynchronous wrote: post #22

Earlier quoted context omitted.

I’ve been amazed at how quickly the open source community has iterated on LLMs and Diffusion models. Goes to show how well open source can work.

Innovation in the tech world is spurred by open access.

Support the open companies. Avoid the closed ones, even if they are fantastic at marketing. ;)

Re: Running Stable Diffusion in 260MB of RAM

#48

Post by xtagon »

speedgoose wrote: post #34

I like the use of a tiny device to generate the images. I was wondering whether the energy consumption per image would be lower, but I did the simple maths and it's not the case. A raspberry pi zero 2W seems to use about 6W under load (source: https://www.cnx-software.com/2021/12/09/raspberry-pi-zero-2-... ) So if it takes 3 hours to generate one picture, that's about 18Wh per image. A Nvidia Tesla or RTX GPU can gen…

Would an accelerator such as the Intel Neural Compute Stick 2 work with this? It can be plugged into a Pi, however I'm not clear on how VRAM works on the compute stick or if it's shared with the host?

Re: Running Stable Diffusion in 260MB of RAM

#49

Post by hospitalJail »

wsgeorge wrote: post #3
boredemployee wrote: post #2

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD. I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

On an Apple M1 with 16gig RAM, without using Pytorch compiled to take advantage of Metal, it could take 12mins to generate an image with a tweet-length prompt. With Metal, it takes less than 60 seconds.

Wtf, my 4 year old, $400 crappy low wattage computer can generate a picture in a minute or two.

DDIM, 12 steps.

Re: Running Stable Diffusion in 260MB of RAM

#50

Post by dheera »

isoprophlex wrote: post #12

Incredible! If only there was some cheap hackable eink frame, you could make a fully self contained artwork from eink panel + rpi that's (slowly) continuously updating itself..!

I made one before (https://dheera.net/projects/einkframe/) that used ShanShui (https://github.com/LingDong-/shan-shui-inf)

I'm thinking of making a Stable Diffusion version of this, and preferably with a larger eInk screen.

Post reply on HN