I don't see any explanation for why they trained 8B instead of 7B.
I thought that If you have a 16GB GPU, you can put 14GB(7B*16bits) model into it, but how does it fit If the model is exactly 16GB?
Meta Llama 3
91–100 of 965 posts
Re: Meta Llama 3
#92If any one is interesting in seeing how 400B model compares with other opensource models, here is a useful chart: https://x.com/natolambert/status/1780993655274414123
[deleted]
Re: Meta Llama 3
#93They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasonin…
ML Twitter was saying that they're working on a 400B parameter version?
Re: Meta Llama 3
#94Re: Meta Llama 3
#95I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone k…
Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant?
I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.
Re: Meta Llama 3
#96https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…
> When the threshold between annoying and profitable is crossed, Meta's lawyers will start sniping and acquiring users of their IP.
I'm curious: given that the model will probably be hosted in a private server, how would meta know or prove that someone is using their model against the license?
Re: Meta Llama 3
#97The model card has the benchmark results relative to other Llama models including Llama 2: https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md... The dramatic performance increase of Llama 3 relative to Llama 2 (even Llama 2 13B!) is very impressive. Doubling the context window to 8k will open a lot of new oppertunities too.
For the instruction tuned models, Llama 3 8B is even significantly better than Llama 2 70B!
To be fair, the Llama 2 instruction tuning was notably bad.
Re: Meta Llama 3
#98https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…
What is "source" regarding an LLM? Public training data and initial parameters?
Not an expert, but often weights are mentioned as not being open sourced.
Happy to get corrected, as I'm not really sure.