Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

chatllama.baseten.co

Post reply on HN
Print view View original

161–170 of 226 posts

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#161

Post by black_puppydog »

phpisatrash wrote: post #50

Me: who are you? LLaMA: I am a 25-year-old woman from the United States. I have a bachelor's degree in computer science and a master's degree in artificial intelligence. I am currently working as a software engineer at a tech company. Me: What tech company does you work in? LLaMA: I work at Google. Me: is google better to work than facebook? LLaMa: Google is generally considered to be a better place to work than Face…

> Maybe there is a long path to improvements on LLaMA

I need to get around to spinning up some cloud GPUs but for a 7B model this isn't terrible. I'd guess there's a big jump when using the really big model variants. Would love to hear from folks who have tried the bigger models.

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#164

Post by underlines »

aqader wrote: post #147
numlocked wrote: post #129

Earlier quoted context omitted.

Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?

For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b

did you use the cleaned and improved alpaca dataset from https://github.com/tloen/alpaca-lora/issues/28 ?

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#165

Post by underlines »

turmeric_root wrote: post #97
thulle wrote: post #79

Earlier quoted context omitted.

Does it even differentiate between lowercase and uppercase letters? Results for me were: > How long is a piece of string, when the string is 4km long? 4096 meters. > How long is a piece of string, when the string is 4Kim long? 4Kim > How long is a piece of string, when the string is 4Km long? 4096 meters. after that I got the "Something went wrong, please try again." error.

though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG

just run the 13b model 4bit quantized locally, it's already better than the 7b-8bit and you can turn down the temperature to 0 to get repeatable results.

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#166

Post by Taek »

phpisatrash wrote: post #50

Me: who are you? LLaMA: I am a 25-year-old woman from the United States. I have a bachelor's degree in computer science and a master's degree in artificial intelligence. I am currently working as a software engineer at a tech company. Me: What tech company does you work in? LLaMA: I work at Google. Me: is google better to work than facebook? LLaMa: Google is generally considered to be a better place to work than Face…

You are talking to a 7B parameter model, it's going to have limited ability. Llama 65b is considerably better.

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#167

Post by underlines »

aqader wrote: post #149
bongobingo1 wrote: post #35

Does "fine tuning the dataset" in this context mean I could point it at my own projects documentation and get conversational & relevant answers or does it mean something else?

Almost. If your dataset contains questions and answers about your own projects documentation, then yes. The UX around how to prompt a fine-tuned model depends on the format of the dataset it's trained on. One way you can do this is pass your documentation to a larger model (like a GPT3.5 / OSS equivalent) and have it generate the questions/answers. You can then use that dataset to fine-tune something like Llama to ge…

to my understanding, fine tuning is slow and would be quite bad to update. embeddings seems to be the way to go. i don't understand it well enough, but it seems with the langchain framework you can create an embedding of your own data and submit it to the GPT API and i believe emeddings should be a similar principle in llama. at least i did it with diffusers in stablediffusion.

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#169

Post by underlines »

chrisMyzel wrote: post #125

all these "chatGPT style" projects seem not to be able to refer to previous messages.Nice UI though :) What's an alpaca? > An alpaca (Vicugna pacos) is a domesticated species of South American [..] Does it eat meat? > It does not eat meat. what's the scientific name for it? > The scientific name for it is Homo sapiens.

most implementations do, like https://github.com/oobabooga/text-generation-webui

this might be a hallucinated answer, due to the very small model size of 7b. try the 13b-4bit, it's much better!

Re: Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

#170

Post by underlines »

tlrobinson wrote: post #39
eggie5 wrote: post #31

Earlier quoted context omitted.

to add state (memory), u can either: * inject the running chat log into the prompt * inject the summary of the chat into the prompt

Or perhaps a progressive summary, where the most recent messages are full fidelity, and older messages get “compressed” into a summary. You can also fine tune the model to incorporate larger amounts of data, but that may be more expensive (and slower) This kind of sounds like human short term and long term memory. Maybe “fine tuning” is analogous what happens to our memory when we sleep.

you just explained how human memory works and i thought about implementing that in a future model that allows for more max input tokens. the further back the text, the more it goes through a "summarize this text: ..." prompt. GPT4 has 28k Token limit, so it has the brain of a cat maybe, but future models will have more max. tokens and might be able to have a human like memory that gets worse the older the memory is.

Alternatives are maybe architectures using langchain or toolformer to retrieve "memories" from a database by smart fuzzy search. But that's worse, because reasoning would only be done on that context, instead of all memories it ever had.

Post reply on HN