11–20 of 58 posts
As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui
Nice stuff all the same.
Earlier quoted context omitted.
It supports local models.
Are there any good local models? Gpt-2 is pants once you've gotten used to 4.
Having fun right now trying to build a model in GDELT, not much luck so far, but I’ve pushed less than 5% of the data through so far.
I’ve also been experimenting on fine-tuning Llama on my personal data archives, which seems promising but it’s pretty expensive to do so. Hoping someone will release a ~13B param model of Llama that they’ve trained with transfer learning from the 65B llama model and other data. FWIW even the 7B llama model running through llama.cpp after being quantized performs (subjectively, but substantially) better than GPT2.
This has been one of my most expensive, but also most rewarding, hobbies thus far.
Earlier quoted context omitted.
Are there any good local models? Gpt-2 is pants once you've gotten used to 4.
GPT2 is ancient news. There are now local running models which, allegedly, can reach the same performance as GPT-4. Look up llama.cpp[1] and all the various community generated models. [1] https://github.com/ggerganov/llama.cpp
As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui
Earlier quoted context omitted.
Are there any good local models? Gpt-2 is pants once you've gotten used to 4.
GPT2 is ancient news. There are now local running models which, allegedly, can reach the same performance as GPT-4. Look up llama.cpp[1] and all the various community generated models. [1] https://github.com/ggerganov/llama.cpp
An LLM playground whose UI you can run on your laptop.
Earlier quoted context omitted.
Are there any good local models? Gpt-2 is pants once you've gotten used to 4.
I don’t know about out of the box, but I’ve been having a great time training my own domain-specific models from scratch and utilizing them locally. It does seem that the more domain specific a task is, far fewer params are needed. Having fun right now trying to build a model in GDELT, not much luck so far, but I’ve pushed less than 5% of the data through so far. I’ve also been experimenting on fine-tuning Llama on m…
Do you train exclusively in the cloud?
An LLM playground whose UI you can run on your laptop.
Its amazing can run Alpaca llama 33B parameters, totally can handle japanese and korean where the earlier ones like 7B parameters could only do english (any other languages was horrible). All able to run on my M1 macbook.
As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui