191–200 of 232 posts
So many so-called overnight AI gurus hyping about their snake-oil product and screaming about 'Meta is dying' [0] and 'It is over for Meta' but little of them actually do research in AI and drive the field forward and this once again shows that Meta has always been a consistent contributor to AI research, especially in vision systems. All we can just do is take, take, take the code. But this time, the code's license…
Earlier quoted context omitted.
You "spend hours messing around" with everything you don't know or understand at first. One could say the same about writing the software itself. At its core Dockerfiles are just shell scripts with worse syntax, so it's not really that much more to learn. Once you get it done once, you don't have to screw around with it anymore, and you have it on any box you want in seconds. In either case you have to spend hours sc…
> Storing data in a database vs in files on disk is like application development 101 It's OK until you're dealing with, say, 130GiB of tensors, on what is effectively a binary blob that needs to be mostly in VRAM somehow. I really don't want to read 130GiB of blobs from a database all the time.
Meta on a roll. any demo on how good the text to speech is ?
I would like to use stuff like this as a side-project. Buy a Nvidia Geforce GPU and stick it into my 24/7 server and play around with it in my free time, to see what can be done. The issue with all these AI models is that there's no information on which GPU is enough for which task. I'm absolutely clueless if a single RTX 4000 SFF with its 20GB VRAM and only 70W of max power usage will be a waste of money, or really…
You can finetune whisper, stable diffusion, and LLM up to about 15B parameters with 24GB VRAM. Which leads you to what hardware to get. Best bang for the $ right now is definitely a used 3090 at ~$700. If you want more than 24GB vram just rent the hardware as it will be cheaper. If you're not willing to drop $700 don't buy anything just rent. I have had decent luck with vast.ai
I would like to use stuff like this as a side-project. Buy a Nvidia Geforce GPU and stick it into my 24/7 server and play around with it in my free time, to see what can be done. The issue with all these AI models is that there's no information on which GPU is enough for which task. I'm absolutely clueless if a single RTX 4000 SFF with its 20GB VRAM and only 70W of max power usage will be a waste of money, or really…
This repo lists very specific VRAM usage for various LLaMA models (w/ group size, and accounting for context window, which is often missing) - this are all 4-bit GPTQ quantized models: https://github.com/turboderp/exllama
Note the latest versions of llama.cpp now have decent GPU support and has both a memory tester and lets you load partial models (n-layers) into your GPU. It inferences about 2X slower than exllama from my testing on a RTX 4090, but still about 6X faster than my CPU (Ryzen 5950X).
Again this is inferencing. For training, pay attention to 4-bit bitsandbytes, coming soon: https://twitter.com/Tim_Dettmers/status/1657010039679512576
Earlier quoted context omitted.
That's a related complaint: everyone continues to insist on releasing papers as PDFs, ignoring the fact that those are still pretty nasty to read on a mobile device. Sure, release a PDF (some people like those), but having an additional responsive web page version of a paper makes research much more readable to the majority of content consumption devices. It's 2023. I'll generally use https://www.arxiv-vanity.com/ to…
Would you like a back rub while I'm at it?
Interesting that there isn’t a common solution for this. I guess it’s rather niche?
I would like to use stuff like this as a side-project. Buy a Nvidia Geforce GPU and stick it into my 24/7 server and play around with it in my free time, to see what can be done. The issue with all these AI models is that there's no information on which GPU is enough for which task. I'm absolutely clueless if a single RTX 4000 SFF with its 20GB VRAM and only 70W of max power usage will be a waste of money, or really…
Imagine if we used these types of models for like 500 years and it locked their vocabulary in time, disallowing any further language blending; then somehow the servers turned off and nobody could communicate across language barriers anymore. Someone should write that down in some sort of short-story involving a really tall structure.
Imagine if someone said this about dictionaries.