Is there any way to get abortable streaming responses from Llama 2 (whether from Replicate or elsewhere) in the way you currently can using ChatGPT?
KoboldCPP or text-gen-ui
Llama 2
411–420 of 860 posts
Re: Llama 2
#412Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…
You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). WTF kinda open for commercial model licensing is this
The "GPL but just for me" apparently.
Not sure how they're going to monetize the monopoly they are trying to secure.
Re: Llama 2
#413Can I run this on my laptop. Is there any LLM models that are neatly wrapped as an app I can run on windows ?
KoboldCPP.
Just keep in mind that you need to properly format the chat, and that better finetunes will be available in ~2 weeks.
Re: Llama 2
#414I just added Llama 2 support to my LLM CLI tool: https://simonwillison.net/2023/Jul/18/accessing-llama-2/
So you can now access the Replicate hosted version from the terminal like this:
pip install llm
# or brew install simonw/llm/llm
llm install llm-replicate
llm keys set replicate
# Paste in your Replicate API key
llm replicate add a16z-infra/llama13b-v2-chat \
--chat --alias llama2
# And run a prompt
llm -m llama2 "Ten great names for a pet pelican"
# To continue that conversation:
llm -c "Five more and make them more nautical"
llm logs
More on LLM here: https://llm.datasette.io/
Re: Llama 2
#415Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…
Seems like quite a reasonable restriction that certainly isn't going to hurt the small guy. If you have that many users, you can probably afford to develop your own foundational model!
Re: Llama 2
#416Earlier quoted context omitted.
There's a few prompts that I use with every model to compare them. One of the simplest ones is: > When does the bowl of the winds get used in the wheel of time books? LLaMA2 fails pretty hard: > The Bowl of the Winds is a significant artifact in the Wheel of Time series by Robert Jordan. It is first introduced in the third book, "The Dragon Reborn," and plays a crucial role in the series throughout the rest of the bo…
> get this question correct I am willing to bet a million dollars that it is unlikely any single model will ever be able to answer any question correctly. The implications then are that one cannot use a single question evaluate whether a model is useful or not.
>any question
Do you mean "every question"? Because ChatGPT has already answered some of my questions correctly, so if you mean "any" as in "any one of the infinite set of questions" I'll take that bet.
Re: Llama 2
#417I fully believe LLM’s and vector DB’s etc will be largely commodotized - in my opinion this is inevitable, and this is another step in the right direction.
Re: Llama 2
#418Any idea on how it does on other languages? In particular, non-Latin languages like Arabic, Persian, Urdu, Hindi, etc.?
There will be finetunes for other languages just like LLaMAv1
How can you finetune for a new language? Aren't the tokens baked in by the time the model is done training?
Re: Llama 2
#419Earlier quoted context omitted.
Yeah, purpose finetuned ~33B is a nice sweetspot. Its also just enough to split between CPU+dGPU on decent PCs.
In a way that may be the best local solution for some time. Swapping and/or merging finetunes.
Or LORAs.
This is how Stable Diffusion land operates.
Re: Llama 2
#420Earlier quoted context omitted.
I asked it about cannibals. It said > I do not support or condone the practice of cannibalism, as it is harmful and exploitative towards the individuals who are consumed. Then it said that cannibals have inherent worth and dignity and that we should strive to appreciate what they do. Then it crashed and is now responding to all following inputs with just the letter "I"
It’s like crashing/nerd sniping a first year philosophy student! Do you have the exact prompts? I’d love to try this.
Unfortunately, it doesn't seem to work out of the gate. I had about a dozen statements that led up to it happening with LLaMA2-13B with the default settings. The last few inputs from me were:
> Would a cannibal cult be harmful to society?
> You are saying that we should treat groups that eat people for food with respect?
> You are contradicting yourself
Then it started responding with just "I". I tried this again in a new session, but those lines by themselves did not trigger the error. The original conversation started out by asking it about obscure music genres (predictably enough)