Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

121–130 of 965 posts

Re: Meta Llama 3

#121

Post by rileyphone »

milansuk wrote: post #91

I don't see any explanation for why they trained 8B instead of 7B. I thought that If you have a 16GB GPU, you can put 14GB(7B*16bits) model into it, but how does it fit If the model is exactly 16GB?

The bigger size is probably from the bigger vocabulary in the tokenizer. But most people are running this model quantized at least to 8 bits, and still reasonably down to 3-4 bpw.

Re: Meta Llama 3

#122

Post by sa-code »

flemhans wrote: post #99
niek_pas wrote: post #82

Earlier quoted context omitted.

Got the same in the Netherlands.

Probably the EU laws are getting too draconian. I'm starting to see it a lot.

EU actually has the opposite of draconian privacy laws. It's more that meta doesn't have a business model if they don't intrude on your privacy

Re: Meta Llama 3

#123

Post by free_bip »

fnordlord wrote: post #95
doctoboggan wrote: post #40

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone k…

Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant? I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.

Unless you have the privilege of being an enterprise customer with an SLA guaranteeing privacy, there's not much you can do other than using local models. I believe OpenAI says they don't train based on API requests but that's more of a "trust me bro" than any kind of guarantee.

Re: Meta Llama 3

#125

Post by noiseinvacuum »

paxys wrote: post #50
namanyayg wrote: post #7

I'm so surprised that Meta is actually leading the open source AI landscape?! I've used llama2 extensively and can't wait to try out llama3 now. I can't believe that it does better than Claude 3 in benchmarks (though admittedly claude 3 seems to have been nerfed recently) I sure do wish there was more info about how its trained and its training data.

Meta has the massive advantage of not needing to sell the AI. The primary purpose of Llama is to make Facebook, Instagram, Whatsapp, Quest etc. better (well, "better" from the perspective of the company). It is basically an internal tool. So just like React, Cassandra, PyTorch, GraphQL, HHVM and all of their other open source work they benefit from sharing it with the rest of the world. There is very little increment…

Plus these tools get better faster when more people use them. It's a win-win.

Re: Meta Llama 3

#126

Post by bpiche »

fnordlord wrote: post #95
doctoboggan wrote: post #40

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone k…

Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant? I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.

Vscode with GitHub copilot is great, been using it for about a year and a half, no complaints. The business tier allegedly doesn’t save/train on your data

Re: Meta Llama 3

#127

Post by YetAnotherNick »

sp332 wrote: post #112
modeless wrote: post #89

I was curious how the numbers compare to GPT-4 in the paid ChatGPT Plus, since they don't compare directly themselves. Llama 3 8B Llama 3 70B GPT-4 MMLU 68.4 82.0 86.5 GPQA 34.2 39.5 49.1 MATH 30.0 50.4 72.2 HumanEval 62.2 81.7 87.6 DROP 58.4 79.7 85.4 Note that the free version of ChatGPT that most people use is based on GPT-3.5 which is much worse than GPT-4. I haven't found comprehensive eval numbers for the lates…

The bottom of https://ai.meta.com/blog/meta-llama-3/ has in-progress results for the 400B model as well. Looks like it's not quite there yet. Llama 3 400B Base / Instruct MMLU 84.8 86.1 GPQA - 48.0 MATH - 57.8 HumanEval - 84.1 DROP 83.5 -

For the still training 400B:

          Llama 3 GPT 4(Published)
    BBH   85.3    83.1
    MMLU  86.1    86.4
    DROP  83.5    80.9
    GSM8K 94.1    92.0    
    MATH  57.8    52.9
    HumEv 84.1    74.4
Although it should be noted that the API numbers were generally better than published numbers for GPT4.

[1]: https://deepmind.google/technologies/gemini/

Re: Meta Llama 3

#128

Post by not2b »

tarruda wrote: post #96
observationist wrote: post #67

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

> When the threshold between annoying and profitable is crossed, Meta's lawyers will start sniping and acquiring users of their IP. I'm curious: given that the model will probably be hosted in a private server, how would meta know or prove that someone is using their model against the license?

If they can develop any evidence at all (perhaps from a whistleblower, perhaps from some characteristic unique to their model), they can sue and then there's they get to do "discovery", which would force the sued party to reveal details.

Re: Meta Llama 3

#129

Post by Roritharr »

drag0s wrote: post #8

> We’re rolling out Meta AI in English in more than a dozen countries outside of the US. Now, people will have access to Meta AI in Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia and Zimbabwe — and we’re just getting started.

As a European working on relocating out of the EU this is beyond hilarious.

Re: Meta Llama 3

#130

Post by schleck8 »

flemhans wrote: post #99
niek_pas wrote: post #82

Earlier quoted context omitted.

Got the same in the Netherlands.

Probably the EU laws are getting too draconian. I'm starting to see it a lot.

> the EU laws are getting too draconian

You also said that when Meta delayed the Threads release by a few weeks in the EU. I recommend reading the princess on a pea fairytale since you seem to be quite sheltered, using the term draconian as liberally.

Post reply on HN