Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Stable Diffusion in 260MB of RAM

github.com

Post reply on HN
Print view View original

61–66 of 66 posts

Re: Running Stable Diffusion in 260MB of RAM

#61

Post by PeterStuer »

speedgoose wrote: post #36
hadlock wrote: post #35

Earlier quoted context omitted.

For on prem use, the up front cost is a lot lower. The A100 that most serious outfits are using runs in the thousands to tens of thousands of dollars per unit with very limited availability. The pi is typically under $75 usd for any variant.

A RTX 4090 has a much better value for stable diffusion but yes if you start to think about cost the pi wins. If you think about availability, I’m not sure.

An RTX 3090 hits the current sweetspot of price/performance for me. Half the throughput of the 4090, but at 1/3rd the cost. (I needed the 24GB VRAM for other LLM projects).

Re: Running Stable Diffusion in 260MB of RAM

#62

Post by luc4sdreyer »

boredemployee wrote: post #15
londons_explore wrote: post #10

Earlier quoted context omitted.

Use a free GPU from google colab and you can do the same in about 15 seconds...

Do you have a google colab link?

On 21 April 2023 Google blocked usage of Stable Diffusion with a free account on colab. You need a paid plan to use it.

Apparently there are ways around it, but I just switched to runpod.io. It's very cheap (around $0.80/h for a 4090 including storage) and having a real terminal is worth it.

Re: Running Stable Diffusion in 260MB of RAM

#63

Post by MorbidCuriosity »

jlokier wrote: post #60
MorbidCuriosity wrote: post #58

Earlier quoted context omitted.

Yes. Infinity is weird.

No, because GPT-4 has finite memory, its context length, and its random number generator for output selection is probably pseudo-random with finite memory. If the random number generator is pseudo-random, this makes GPT-4 a deterministic finite-state machine, and the output sequence does not necessarily contain all possible subsequences no matter how many times the monkey types a new random key. Put differently, some…

That's a good point, I hadn't considered the limits of GPT's memory.

Re: Running Stable Diffusion in 260MB of RAM

#64

Post by arvinsim »

PeterStuer wrote: post #61
speedgoose wrote: post #36

Earlier quoted context omitted.

A RTX 4090 has a much better value for stable diffusion but yes if you start to think about cost the pi wins. If you think about availability, I’m not sure.

An RTX 3090 hits the current sweetspot of price/performance for me. Half the throughput of the 4090, but at 1/3rd the cost. (I needed the 24GB VRAM for other LLM projects).

Is this brand new or used?

Re: Running Stable Diffusion in 260MB of RAM

#65

Post by adventured »

arvinsim wrote: post #64
PeterStuer wrote: post #61

Earlier quoted context omitted.

An RTX 3090 hits the current sweetspot of price/performance for me. Half the throughput of the 4090, but at 1/3rd the cost. (I needed the 24GB VRAM for other LLM projects).

Is this brand new or used?

Used is the only way to get a 3090 for ~$650-$750 (they're not hard to find on eBay in that general price area).

Re: Running Stable Diffusion in 260MB of RAM

#66

Post by PeterStuer »

arvinsim wrote: post #64
PeterStuer wrote: post #61

Earlier quoted context omitted.

An RTX 3090 hits the current sweetspot of price/performance for me. Half the throughput of the 4090, but at 1/3rd the cost. (I needed the 24GB VRAM for other LLM projects).

Is this brand new or used?

Used from ebay in my case.
Post reply on HN