Is there some single page that keeps a running status of the various LLVM's and the software to make them runnable on consumer hardware?
State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
31–40 of 143 posts
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#32Earlier quoted context omitted.
If you follow this command in their instruction, the delta will be automatically downloaded and applied to the base model. https://github.com/lm-sys/FastChat#vicuna-13b : `python3 -m fastchat.model.apply_delta --base /path/to/llama-13b --target /output/path/to/vicuna-13b --delta lmsys/vicuna-13b-delta-v0`
I may have missed the detail, but it also expects the pytorch conversion rather than original LLaMa model.
Yes, you need to convert the original LLaMA model to the huggingface format, according to https://github.com/lm-sys/FastChat#vicuna-weights and https://huggingface.co/docs/transformers/main/model_doc/llam...
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#33Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
It's fairly similar to a ROM patch in the video game space, which has mostly stood the test of time.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#34It's actually very impressive. I gave it the task of converting a query and an OpenAPI spec into an API call, and it worked! I've not been succesful in getting GPT-3.5 to do this without rambling on about the reasoning for its decision.
Wow, that is very interesting. Would you mind sharing the prompt you used to query the model?
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#35Is there some single page that keeps a running status of the various LLVM's and the software to make them runnable on consumer hardware?
Hi! Funnily enough I couldn't find much on it either, so that's exactly what I've been working on for the past few months: just in case this kind of question got asked.
I've recently opened a GitHub repository which includes information for both AI model series[0] and frontends you can use to run them[1]. I've wrote a Reddit post beforehand that's messier, but a lot more technical[2].
I try to keep them as up-to-date as possible, but I might've missed something or my info may not be completely accurate. It's mostly to help get people's feet wet.
[0] - https://github.com/Crataco/ai-guide/blob/main/guide/models.m...
[1] - https://github.com/Crataco/ai-guide/blob/main/guide/frontend...
[2] - https://old.reddit.com/user/Crataco/comments/zuowi9/opensour...
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#36Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#37From the git repo:
> This conversion command needs around 60 GB of CPU RAM.
Ok. I don't have that. Has/will someone release the full weights with the deltas applied?
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#38Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
(I work on llama-dl.)
We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782
I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individual hackers.
The DMCAs were also launched by a third party law firm, not Meta themselves, so there’s a bit of “left hand doesn’t know what the right hand is doing” in all of this.
I’ll keep everyone updated. For now, hack freely.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#39Earlier quoted context omitted.
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…
keep up god's work!
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#40Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?
I got the 64gb MacBook Pro but already realizing the 96gb laptop would have made sense now - I got it in Jan right before all the ai crazy really lite up - distinctly remember thinking who would ever need more then 64gb of ram…