Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

151–160 of 965 posts

Re: Meta Llama 3

#151

Post by PokeyCat »

Would love to experiment with this for work, but the following clause in the license (notably absent in the Llama 2 license) would make this really hard:

> i. If you distribute or make available the Llama Materials (or any derivative works thereof), or a product or service that uses any of them, including another AI model, you shall (A) provide a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Meta Llama 3” on a related website, user interface, blogpost, about page, or product documentation. If you use the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama 3” at the beginning of any such AI model name.

Really impressive HumanEval results for the 8B model though, would love to plug this into Continue for tab completion since the current benchmark numbers for Llama 3 8B blow every other 8B model out of the water

Re: Meta Llama 3

#152

Post by danenania »

fnordlord wrote: post #95
doctoboggan wrote: post #40

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone k…

Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant? I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.

I'm building Plandex (https://github.com/plandex-ai/plandex), a terminal-based AI coding tool which currently uses the OpenAI api--I'm working on support for Anthropic and OSS models right now and hoping I can ship it later today.

You can self-host it so that data is only going to the model provider (i.e. OpenAI) and nowhere else, and it gives you fine-grained control of context, so you can pick and choose exactly which files you want to load in. It's not going to pull in anything in the background that you don't want uploaded.

There's a contributor working on integration with local models and making some progress, so that will likely be an option in the future as well, but for now it should at least be a pretty big improvement for you compared to the copy-paste heavy ChatGPT workflow.

Re: Meta Llama 3

#153

Post by sidcool »

drag0s wrote: post #8

> We’re rolling out Meta AI in English in more than a dozen countries outside of the US. Now, people will have access to Meta AI in Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia and Zimbabwe — and we’re just getting started.

Not available in India.

Re: Meta Llama 3

#154

Post by andy99 »

gliched_robot wrote: post #105
modeless wrote: post #89

I was curious how the numbers compare to GPT-4 in the paid ChatGPT Plus, since they don't compare directly themselves. Llama 3 8B Llama 3 70B GPT-4 MMLU 68.4 82.0 86.5 GPQA 34.2 39.5 49.1 MATH 30.0 50.4 72.2 HumanEval 62.2 81.7 87.6 DROP 58.4 79.7 85.4 Note that the free version of ChatGPT that most people use is based on GPT-3.5 which is much worse than GPT-4. I haven't found comprehensive eval numbers for the lates…

Wild considering, GPT-4 is 1.8T.

Once benchmarks exist for a while, they become meaningless - even if it's not specifically training on the test set, actions (what used to be called "graduate student descent") end up optimizing new models towards overfitting on benchmark tasks.

Re: Meta Llama 3

#155

Post by Davidzheng »

dmarchand90 wrote: post #48
hermesheet wrote: post #36

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

Where does it say much better than gpt4 for the 400B model?

It doesn't ....

Re: Meta Llama 3

#156

Post by azinman2 »

namanyayg wrote: post #7

I'm so surprised that Meta is actually leading the open source AI landscape?! I've used llama2 extensively and can't wait to try out llama3 now. I can't believe that it does better than Claude 3 in benchmarks (though admittedly claude 3 seems to have been nerfed recently) I sure do wish there was more info about how its trained and its training data.

Llama* aren't open source. They just have a relatively open license except for competitors.

Re: Meta Llama 3

#157

Post by PokeyCat »

verdverm wrote: post #88

Initial observations from the Meta Chat UI... 1. fast 2. less censored than other mainstream models 3. has current data, cites sources I asked about Trump's trial and it was happy to answer. It has info that is hours old --- Five jurors have been selected so far for the hush money case against former President Donald Trump ¹. Seven jurors were originally selected, but two were dismissed, one for concerns about her im…

It's likely RAG / augmented with web data. Would be interested if local execution returned the same results.

Re: Meta Llama 3

#158

Post by mananaysiempre »

blackeyeblitzar wrote: post #136
CuriouslyC wrote: post #38

Earlier quoted context omitted.

They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasonin…

Llama is open weight, not open source. They don’t release all the things you need to reproduce their weights.

Not really that either, if we assume that “open weight” means something similar to the standard meaning of “open source”—section 2 of the license discriminates against some users, and the entirety of the AUP against some uses, in contravention of FSD #0 (“The freedom to run the program as you wish, for any purpose”) as well as DFSG #5&6 = OSD #5&6 (“No Discrimination Against Persons or Groups” and “... Fields of Endeavor”, the text under those titles is identical in both cases). Section 7 of the license is a choice of jurisdiction, which (in addition to being void in many places) I believe was considered to be against or at least skirting the DFSG in other licenses. At best it’s weight-available and redistributable.

Re: Meta Llama 3

#159

Post by cjbprime »

refulgentis wrote: post #141
oersted wrote: post #116

Earlier quoted context omitted.

Where did you find this number? Not doubting it, just want to get a better idea of how precise the estimate may be.

It's a really funny story that I comment about at least once a week because it drives me nuts. 1. After ChatGPT release, Twitter spam from influencers about chatGPT is one billion and GPT-4 is 1 trillion . 2. Semianalysis publishes a blog post claiming 1.8T sourced from insiders. 3. The way info diffusion works these days, everyone heard from someone else other than Semianalysis. 4. Up until about a month ago, you co…

You're ignoring geohot, who is a credible source (is an active researcher himself, is very well-connected) and gave more details (MoE with 8 experts, when no-one else was doing production MoE yet) than the Twitter spam.

Re: Meta Llama 3

#160

Post by jph00 »

observationist wrote: post #67

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

> "Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing."

You seem to be making claims that have little connection to the actual license.

The license states you can't use the model if, at the time Llama 3 was released, you had >700 million customers. It also says you can't use it for illegal/military/etc uses. Other than that, you can use it as you wish.

Post reply on HN