601–610 of 719 posts
Earlier quoted context omitted.
4chan has been in full doomer mode for years. It didn't used to be, from what I remember, though I was never an active denizen. I'd love to understand the sociology behind the change in vibe that happened there.
Too much anime and weed
Earlier quoted context omitted.
The checkpoint for the 7B parameter model is 13.5GB, so maybe? Larger models are multiple chunks at 13.6GB each or 16.3GB each. I am hoping I will be able to run on my 16GB Vram but I don't know how much overhead is needed. Maybe people on reddit will do their tricks and squeeze the models in to smaller cards. EDIT: There seems to be a lot of overhead. Here someone struggles to fit the 7B parameter model (13.5GB chec…
Following up. After rebooting in to GUI that was enough to get it to fit, I guess xorg just accumulated some cruft in my last boot. So I can run it alongside gnome. nvidia-smi reports this model is using 15475MiB after changing the max batch size from 32 to 8 (see link in above post) As others have stated someone may have injected unknown code in to the pickled checkpoint, so I recommend running this in docker. I use…
Earlier quoted context omitted.
That just shows their bias. If the code is complete, it's only a matter of time to figure out how to build it. May not take hours, but determined engineer should be able to figure it out.
I think you're being naive. Microsoft has spent 1000s of engineering years on their build system. You aren't going to just replicate that in a week.
Earlier quoted context omitted.
Are you saying they are like compact memoizers? What Stable Diffusion can fit into that model is amazing.
When SD1.4 dropped, someone here described how those models are a form of lossy compression.
Earlier quoted context omitted.
Following up. After rebooting in to GUI that was enough to get it to fit, I guess xorg just accumulated some cruft in my last boot. So I can run it alongside gnome. nvidia-smi reports this model is using 15475MiB after changing the max batch size from 32 to 8 (see link in above post) As others have stated someone may have injected unknown code in to the pickled checkpoint, so I recommend running this in docker. I use…
I was able to run 7B on a CPU, inferring several words per second: https://github.com/markasoftware/llama-cpu
opt-175B weights are already openly available as I understand. Hugging-face also has openly available weights for a 176B parameter LLM called Bloom. Is LLAMA offering something over and above these?
opt-175B doesn't exist; the largest one is opt-66B. And, at least in the tests I've run (not with the biggest one, but only up to a dozen billion parameters), all the opt models severely underperform with respect to even much smaller models. To the point that the launch of OPT (before BLOOM) was literally advertised as "the biggest OpenSource language model released to date", because they couldn't push on much else.…
https://ai.facebook.com/blog/democratizing-access-to-large-s...
Earlier quoted context omitted.
> It's also unique in that humans could destroy the biosphere if we wanted to - that was never possible before the mid-20th. It's not possible now either. If all of humanity's efforts were devoted to this task, they would not even make a noticeable difference.
My assessment for most of my life has been if most of the world's ~10k 'strategic' megaton-scale warheads exploded in air over Earth's major cities it would kick up enough dust to kill the sun for several years, which would kill off a large fraction of Earth's flora and fauna, akin to a major volcanic eruption or asteroid collision. There would still be life of the smaller sort, and deep in the oceans of course. Only…
Earlier quoted context omitted.
When SD1.4 dropped, someone here described how those models are a form of lossy compression.
Back when wavelet compression was still being developed, there was a joke that the best compression algorithm is "give an image to a grad student and tell them to figure out the best transform".
Fractal Image Compression, https://en.wikipedia.org/wiki/Michael_Barnsley https://www.abebooks.com/servlet/BookDetailsPL?bi=3131987970...
Earlier quoted context omitted.
No need to spend $30k, use Azure or AWS.
Yep. It’s expensive to spin up an A100 80GB instance but not THAT expensive. Oracles cloud offering (first thing to show up in google search I know you probably won’t use them and it seems extra expensive) is $4.00 per hour. If you are motivated to screw around with this stuff there’s definitely options.
Earlier quoted context omitted.
Yep. It’s expensive to spin up an A100 80GB instance but not THAT expensive. Oracles cloud offering (first thing to show up in google search I know you probably won’t use them and it seems extra expensive) is $4.00 per hour. If you are motivated to screw around with this stuff there’s definitely options.
GCP spot price for A100 80g gpu is only $1.25 and they give you $300 of credit when you open a new acc