Their 7b model performs much better than the 13b model I have running locally with alpaca.cpp. I think this is because (looking at their code) they stick with the float16 parameters while alpaca.cpp is using the quantizized 4bit parameters. The difference is quite noticeable.
Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
141–150 of 226 posts
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#142anyone know where to get the latest information on progress on training Alpaca on the 65B llama model? I've seen a few reports on Twitter etc of people running the llama CPP script with the 65B model on an M1 macbook, but would be good to see more detailed instructions on how to do it and a writeup of how well it works compared to the smaller models etc. I'm happy to help out if anyone is working on this, either with…
I found what I was looking for here [0]. Looking forward to seeing Alpaca on top of it - so far it seems pretty underwhelming compared to GPT, and often just regurgitates 4chan-looking stuff (with timestamps and usernames) like earlier generative models did.
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#143Interestingly, when asking questions in French or Spanish, it replies accordingly but in English. > Est-ce que le langage Go est plus utilisé que Rust? Yes, Go is more widely used than Rust. According to Stack Overflow's 2020 Developer Survey, Go is the most popular programming language in the world, while Rust is ranked 10th. > Cual es el nombre del rey de España? The name of the king of Spain is Felipe VI.
Isn't that absolutely fascinating? For me at least it is, it also seems so effortless
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#144It seems it gets stuck in pseudo infinite loops sometimes. Can you provide a step by step guide on how to get dressed for a five years old? 1. Start by putting on underwear and socks. 2. Put on a t-shirt. 3. Put on pants or shorts. 4. Put on shoes or sandals. 5. Put on a jacket or sweater. 6. Put on a hat or scarf. 7. Put on gloves or mittens. 8. Put on a coat. 9. Put on a hat or scarf. 10. Put on shoes or sandals. 1…
I found these loops also for poems and source code with loops in two out of three attempts.
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#145Rap battle between ChatGPT and ChatLLaMA (made by copy/pasting responses from one to the other). Note that ChatLLaMA manages to confuse ChatGPT about who is who. == ChatLLaMA: I'm gonna drop some rhymes that'll blow your mind. I'm gonna show you what I'm made of. (note: it repeats "I'm gonna show you what I'm made of." dozens of times but I edited those out). === ChatGPT: Alright ChatLLaMA, I'm down for this rap batt…
Did ChatLLaMA just echo back what ChatGPT spit out? That's not particularly promising
I did exactly that. It makes sense given that it's a small model which isn't trained on RLHF.
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#146Wow sorry y'all, we didn't expect this to take off so quickly. Working on getting this scaled up now so everyone can play!
Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?
[deleted]
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#147Wow sorry y'all, we didn't expect this to take off so quickly. Working on getting this scaled up now so everyone can play!
Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?
For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b
We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#148This performs considerably better than simply running llama.cpp. I've tested with 7b, 13b and 30b parameters alpaca models locally and I couldn't get them to produce accurate instructions to cut a video with ffmpeg, but this one does. Wonder if it's the 4bit quantization.
Yeah, this is running in 8bit mode. The 30b 8bit version we released seems to do a lot better but it requires significantly more compute.
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#149Does "fine tuning the dataset" in this context mean I could point it at my own projects documentation and get conversational & relevant answers or does it mean something else?
Almost. If your dataset contains questions and answers about your own projects documentation, then yes. The UX around how to prompt a fine-tuned model depends on the format of the dataset it's trained on.
One way you can do this is pass your documentation to a larger model (like a GPT3.5 / OSS equivalent) and have it generate the questions/answers. You can then use that dataset to fine-tune something like Llama to get conversation / relevant answers.
Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
#150How much better would fine tuning be vs few shot learning over own corpus of text?
Depends on the model size. A model like GPT3 that has hundreds of billions of paramaters, you can do few-shot learning with. You'll still pay for the tokens processed and it'll at least linearly increase response times the larger your input is.
Fine-tuning can get you similar results on smaller / faster models. The downside is you have to craft the dataset in the right way. There are trade-offs to both approaches but fwiw, I don't think Alpaca-7b can do few-shot learning.