“In the coming months, we expect to introduce new capabilities, longer context windows, additional model sizes, and enhanced performance, and we’ll share the Llama 3 research paper.”
Meta Llama 3
31–40 of 965 posts
Re: Meta Llama 3
#32* 8k context length * Llama3 8B beats Mistral 7B * Llama3 70B beats Claude 3 Sonnet and Gemini Pro 1.5
* beats in some cases, losses in some cases (vs gemini pro) and has only 8k context window.
Re: Meta Llama 3
#33Earlier quoted context omitted.
Why do people keep saying that Claude3 has been nerfed? Their CTO has said on Twitter multiple times that not a single byte has been changed since its launch, so I'm curious why I keep hearing this. edit: having trouble finding the tweet I saw recently, it might have been from their lead engineer and not the CTO.
Why would the CTO/lead engineer admit that they nerfed the model even if they did? It’s all closed, how does admitting it benefit them? I would much rather trust the people using it everyday.
It's not a random sample of people. You're sampling the 10 most noisy people out of a million users, and those 10 people could be mistaken.
Claude 3 hasn't dropped Elo on the lmsys leaderboard which supports the CTO's claim.
Re: Meta Llama 3
#34Zuck has an interview out for it as well, https://twitter.com/dwarkesh_sp/status/1780990840179187715
[deleted]
Re: Meta Llama 3
#35I'm so surprised that Meta is actually leading the open source AI landscape?! I've used llama2 extensively and can't wait to try out llama3 now. I can't believe that it does better than Claude 3 in benchmarks (though admittedly claude 3 seems to have been nerfed recently) I sure do wish there was more info about how its trained and its training data.
Better than Claude 3 Sonnet but Claude 3 Opus is significantly more powerful, albeit I'm not sure how they compare accounting for parameter size.
Do they publish the parameter size for Sonnet and Opus?
Re: Meta Llama 3
#36Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/
Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!
Re: Meta Llama 3
#37The model card has the benchmark results relative to other Llama models including Llama 2: https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md... The dramatic performance increase of Llama 3 relative to Llama 2 (even Llama 2 13B!) is very impressive. Doubling the context window to 8k will open a lot of new oppertunities too.
Disappointed to note that the 8k context length is far short of Mixtral 8x22B's 64k context length. Still, the published performance metrics are impressive. Kudos to Meta for putting these models out there.
They’re going to increase the context window
https://www.threads.net/@zuck/post/C56MOZ3xdHI/?xmt=AQGzjzaz...
Re: Meta Llama 3
#38They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasoning benchmarks.
Since llama is open source, we're going to see fine tunes and LoRAs though, unlike opus.
Re: Meta Llama 3
#39Is there a download link for this model like LLAMA2 or is it going to be exclusively owned and operated by Meta this time?
Re: Meta Llama 3
#40I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations.
However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3).
Does anyone know if there is any model out there that can be run locally and compete with GPT4-turbo? Or am I asking for something that is impossible?