551–560 of 860 posts
Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…
Earlier quoted context omitted.
Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.
There's a few prompts that I use with every model to compare them. One of the simplest ones is: > When does the bowl of the winds get used in the wheel of time books? LLaMA2 fails pretty hard: > The Bowl of the Winds is a significant artifact in the Wheel of Time series by Robert Jordan. It is first introduced in the third book, "The Dragon Reborn," and plays a crucial role in the series throughout the rest of the bo…
Earlier quoted context omitted.
The impact of this, I think is that the LLM is right, hippos do swim, because that's how people commonly describe what hippos do, such that it gets into the training set
That's just stupid talk. It either swims or it doesnt. A drowning hippo isn't going to wish itself to float.
Correct, it swims.
>A drowning hippo isn't going to wish itself to float.
A drowning hippo probably wishes it can float, much like a drowning person wishes they can float.
Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…
Lately I’ve been wondering if a license similar to this but instead based on market cap could be a way to monetize open source projects E.g. 100k/year for each trillion in market cap, updated yearly. First trillion is free
Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…
Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…
Earlier quoted context omitted.
Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.
Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despi…
Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…
Earlier quoted context omitted.
I’m not worried about the semantics if it is free and available for commercial use too I’m fine just calling “a license”
It's disappointing that you're stuck using LLaMA at Meta's pleasure for their approved application. I was hoping they would show some leadership and release this under the same terms (Apache 2.0) as PyTorch and their other models, but they've chosen to go this route now which sets a horrible precedent. A future where you can only do what FAANG wants you to is pretty grim even if most of the restrictions sound benign…
Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…
Your Llama2 MMLU figure is wrong
I see different figures in different places, no idea what's right.