141–150 of 185 posts
Earlier quoted context omitted.
I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."
Well... it's not wrong...
Earlier quoted context omitted.
I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We released the demo first to get some early feedback on the model.
Has LoRA been considered as possible alternative for finetuning on your dataset? In that case releasing the 'diff' against the LLaMA weights would be simpler to work with.
> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…
Train it on what? I want host our own AI, have it injest our docs, and answer questions about them. What is the best tool today to do that?
This project might be the best option where you can finetune an LLM on your data and keep the model yourself
Earlier quoted context omitted.
They absolutely did not spend millions to train it. Credible estimates place the cost for an entity like Meta at about 30-100k, probably less since Meta likely owns the 256 A100x8s needed to train it. Even as an individual, it wouldn't cost you anywhere near a million if you only trained 13B and took advantage of volume pricing.
I don’t think it’s fair to just ignore the capex part of the model training costs. If we take AWS pricing, the 21 days of training for 65B cited in the llama paper would cost 2.6m at reserved prices. While there’s a lot of AWS profit there, it’s a reasonable first approximation of the TCO of that hardware. Even if real TCO is a third, that’s still nearly a million to train 65B, never mind the staff costs.
>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the man…
Q: We need you to draw seven red lines. All of them strictly perpendicular; some with green ink and some with transparent. Can you do that? A: I'm sorry, but I am a text-based AI model and I don't have the ability to physically draw lines or use different colors of ink. However, I can help you visualize this concept by describing the red lines and the different ink colors. A red line with green ink: Imagine a traditi…
Earlier quoted context omitted.
We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?
It is mainly because of the legal issues caused by the license of llama model weights. We need to figure it out with Meta's llama team before releasing.
I can't understand how something called "a language model" can evaluate some input text. ChatGPT4 seems like it understands the text of the first assistance and properly assesses that it wasn't actually a description of the trip. Similarly, the other day I asked it to explain a hard-to-understand philosophy quote and it explained it properly (I believe). Can someone explain how they do this? It seems magical.
Yes, this is GPT-2 not 4 and it‘s not the RL-trained Chat, only the GPT model and it‘s basically only the inference part, not the training loop and it‘s somewhat simplified.
Still, take a good look.
That‘s essentially what it is on a single sheet of paper.
There is nothing specifically about language in „language model“, we just call it that. Better to call it just LLM.
Nobody knows exactly what it learns, although there would be ways to poke around given some research programs. But it seems like the interest in that is limited currently, everyone is busy with improving it or with applications.
Perhaps the answer is that we overestimated what a mind is. It‘s like we used to ask what life is and it turned out that there is nothing special about life, not even the DNA is controlling anything. It‘s merely a chemical process, even though a complex process.
Personally, I believe there is something special about consciousness and I believe these systems are not conscious. And I believe they are currently not flexible enough to adapt to new circumstances, unlike a human brain. They don‘t currently make much real progress. But that may change.
It's hilarious to see how these models can hallucinate stuff that doesn't exist... Q: What is the meaning of ABBA's song "I have a dream"? A: "I Have a Dream" is a song by the Swedish pop group ABBA, released in 1970. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first hit single. The song is a ballad that talks about a person's dream of finding love and happiness. The lyrics descr…
I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."
>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the man…