Live data from Hacker News

Meta Llama 3

llama.meta.com

81–90 of 965 posts

Re: Meta Llama 3

#81
post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

[flagged]

Sorry, still too sexy. Can’t have that.

Re: Meta Llama 3

#82
post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

> Meta AI isn't available yet in your country Where is it available? I got this in Norway.

Got the same in the Netherlands.

Re: Meta Llama 3

#84
post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

They also stated that they are still training larger variants that will be more competitive:

> Our largest models are over 400B parameters and, while these models are still training, our team is excited about how they’re trending. Over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronger overall capabilities.

Re: Meta Llama 3

#85
post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

And they even allow you to use it without logging in. Didnt expect that from Meta.

Not in the EU though

Re: Meta Llama 3

#86

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

Is it decentralized? You can run it multiple places I guess, but it’s only available from one place.

And it’s not open source.

Re: Meta Llama 3

#87
Quick thoughts -

Major arch changes are not that major, mostly GQA and tokenizer improvements. Tokenizer improvement is a under-explored domain IMO.

15T tokens is a ton!

400B model performance looks great, can’t wait for that to be released. Might be time to invest in a Mac studio!

OpenAI probably needs to release GPT-5 soon to convince people they are still staying ahead.

Re: Meta Llama 3

#88
Initial observations from the Meta Chat UI...

1. fast

2. less censored than other mainstream models

3. has current data, cites sources

I asked about Trump's trial and it was happy to answer. It has info that is hours old

---

Five jurors have been selected so far for the hush money case against former President Donald Trump ¹. Seven jurors were originally selected, but two were dismissed, one for concerns about her impartiality and the other for inaccuracies in his responses during questioning ².

Re: Meta Llama 3

#89
I was curious how the numbers compare to GPT-4 in the paid ChatGPT Plus, since they don't compare directly themselves.

           Llama 3 8B Llama 3 70B GPT-4
 MMLU      68.4       82.0        86.5
 GPQA      34.2       39.5        49.1
 MATH      30.0       50.4        72.2
 HumanEval 62.2       81.7        87.6
 DROP      58.4       79.7        85.4
Note that the free version of ChatGPT that most people use is based on GPT-3.5 which is much worse than GPT-4. I haven't found comprehensive eval numbers for the latest GPT-3.5, however I believe Llama 3 70B handily beats it and even the 8B is close. It's very exciting to have models this good that you can run locally and modify!

GPT-4 numbers from from https://github.com/openai/simple-evals gpt-4-turbo-2024-04-09 (chatgpt)

Re: Meta Llama 3

#90

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

Models are mostly fungible, if meta decided to play games it's not too hard to switch models. I think this is mostly a CYA play.
Post reply on HN