this space moves impossibly fast...
are all open models still just derivatives of llama?
Meta Llama 3
51–60 of 965 posts
Re: Meta Llama 3
#52Looks like 8K context length. Seems to compare well against Gemini Pro 1.5 and Claude 3 Sonnet according to the included benchmarks.
If it's limited to 8k context length then it's not competing with sonnet at all IMO. Sonnet has a 200k context length and it's decent at pulling stuff from it, with just an 8k context length this model won't be great for RAG applications, instead it'll be used for chat and transforming data from one type to another.
They explain that they will be releasing longer context lengths in the future.
It’s better to make your RAG system work well on small context first anyway.
Re: Meta Llama 3
#53Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!
that's very exciting. are you quoting same benchmark comparisons?
Re: Meta Llama 3
#54Earlier quoted context omitted.
It's not a random sample of people. You're sampling the 10 most noisy people out of a million users, and those 10 people could be mistaken. Claude 3 hasn't dropped Elo on the lmsys leaderboard which supports the CTO's claim.
Beyond that, to people who interact with the models regularly the "nerf" issue is pretty obvious. It was pretty clear when a new model rollout caused ChatGPT4 to try and stick to the "leadup, answer, explanation" response model and also start to get lazy about longer responses.
That's a different company's model, so while it may have been obvious it is not relevant to whether Claude 3 has been nerfed or not is it?
Re: Meta Llama 3
#55They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
[flagged]
Re: Meta Llama 3
#56It still say it is using llama2 if you ask it
Re: Meta Llama 3
#57Last week, I made the explicit decision to circle through paid subscriptions of various providers every few months, instead of sticking to just ChatGPT.
Glad to see Meta.ai on the list, in addition to Claude, Gemini, Mistral and Perplexity.
Re: Meta Llama 3
#58Zuck has an interview out for it as well, https://twitter.com/dwarkesh_sp/status/1780990840179187715
I can't express how good Dwarkesh's podcast is in general.
Re: Meta Llama 3
#59They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
[deleted]
Re: Meta Llama 3
#60They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
> Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model
Yeah, almost like comparing a 70b model with a 1.8 trillion parameter model doesn't make any sense when you have a 400b model pending release.