bot: https://meta.ai/
Meta Llama 3
261–270 of 965 posts
Re: Meta Llama 3
#262Re: Meta Llama 3
#263I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…
You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
Depends on your size threshhold. For anything beyond 100 bn in market cap certainly. There is some relatively large companies with a similar flair though, like Cohere and obviously Mistral.
Re: Meta Llama 3
#264Re: Meta Llama 3
#265https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…
> " Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing. " You seem to be making claims that have little connection to the actual license. The license states yo…
That "etc" is doing a lot of work here. The point of OSI licenses like MIT, Apache 2.0 is to remove the "etc". The licensing company gives up its right to impose acceptable use policies. More restrictive, but still OSI approved, licenses are as clear as they possibly can be about allowed uses and the language is as unambiguous as possible. Neither is the case for the Llama AUP.
Re: Meta Llama 3
#266How does it make monetary sense to release open source models? AFAIK it's very expensive to train them. Do Meta/Mistral have any plans to monetize them?
Meta (iirc Zuck in some interview) has stated before that monetizing the models directly isn't part of their plan, they intend to profit from platforms that are built around them, e.g. targeting ads better by having a model look at people's messenger history. It's in their interest to have the community do free work to improve their architecture. Mistral is monetizing them directly, they sell API access.
He just posted an interview today.
Re: Meta Llama 3
#267I'm a big fan of various AI companies taking different approaches. OpenAI keeping it close to their hearts but have great developer apis. Meta and Mistral going open weights + open code. Anthropic and Claude doing their thing.
Competition is a beautiful thing.
I am half excited and half scared that AGI is our generation's space war.
I hope we can solve the big human problems, instead of more scammy ads and videos.
So far AI has been more hype than substance.
Re: Meta Llama 3
#268Earlier quoted context omitted.
Not an expert, but often weights are mentioned as not being open sourced. Happy to get corrected, as I'm not really sure.
Weights aren’t source because the goal of having open source software is that you can know how the software you’re consuming works, and you can produce the final software (the executable) using the source yourself. When you only have weights, you are getting something like the executable. Sure you can tweak it, but you don’t have the things you need to reproduce it or to examine how it works and validate it for your…
I don't think the previous commenter was saying that it's okay to only release the weights.
Re: Meta Llama 3
#269Earlier quoted context omitted.
For the still training 400B: Llama 3 GPT 4(Published) BBH 85.3 83.1 MMLU 86.1 86.4 DROP 83.5 80.9 GSM8K 94.1 92.0 MATH 57.8 52.9 HumEv 84.1 74.4 Although it should be noted that the API numbers were generally better than published numbers for GPT4. [1]: https://deepmind.google/technologies/gemini/
Hm, how much VRAM would this take to run?
Back of the envelope, maybe 0.75TB? More than you have, probably ...
Re: Meta Llama 3
#270Earlier quoted context omitted.
Lex walked so that Dwarkesh could run. He runs the best AI podcast around right now, by a long shot.
I agree that it is the best AI podcast. I do have a few gripes though, which might just be from personal preference. A lot of the time the language used by both the host and the guests is unnecessarily obtuse. Also the host is biased towards being optimistic about LLMs leading to AGI, and so he doesn't probe guests deep enough about that, more than just asking something along the lines of "Do you think next token pre…
There's a difference to being a good chatshow/podcast host and a journalist holding someone's feet to the fire!
Dwarkesh is excellent at what he does - lots of research beforehand (which is how he lands these great guests), but then lets the guest do most of the talking, and encourages them to expand on what they are saying.
It you are critisizing the guest or giving them too much push back, then they are going to clam up and you won't get the best out of them.