Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

181–190 of 965 posts

Re: Meta Llama 3

#181

Post by dudus »

reilly3000 wrote: post #162

I’m impressed by the benchmarks but really intrigued by the press release with the example prompt ~”Tell me some concerts I can go to on Saturday”. Clearly they are able to add their Meta data to context, but are they also crawling the web? Could this be a surface to exfiltrate Meta data in ways that scraping/ APIs cannot?

They are not exposing their super secret trove of data here. Meta can easily control a what the model can or cannot use.

It may have given it the ability to crawl the web and use datasets but only on the hosted version since it would be using tools through func calling.

Re: Meta Llama 3

#182

Post by moffkalast »

doctoboggan wrote: post #40

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone k…

You're asking for something that doesn't exist yet. Command R+, WizardLM-2-8x22B probably come closest.

Re: Meta Llama 3

#185

Post by visarga »

resource_waste wrote: post #55
bbig wrote: post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

[flagged]

GPT-3.5 rejected to extract data from a German receipt because it contained "Women's Sportswear", sent back a "medium" severity sexual content rating. That was an API call, which should be less restrictive.

Re: Meta Llama 3

#186

Post by oliwary »

YetAnotherNick wrote: post #127
sp332 wrote: post #112

Earlier quoted context omitted.

The bottom of https://ai.meta.com/blog/meta-llama-3/ has in-progress results for the 400B model as well. Looks like it's not quite there yet. Llama 3 400B Base / Instruct MMLU 84.8 86.1 GPQA - 48.0 MATH - 57.8 HumanEval - 84.1 DROP 83.5 -

For the still training 400B: Llama 3 GPT 4(Published) BBH 85.3 83.1 MMLU 86.1 86.4 DROP 83.5 80.9 GSM8K 94.1 92.0 MATH 57.8 52.9 HumEv 84.1 74.4 Although it should be noted that the API numbers were generally better than published numbers for GPT4. [1]: https://deepmind.google/technologies/gemini/

Wild! So if this indeed holds up, it looks like OpenAI were about a year ahead when GPT-4 was released, compared to the open source world. However, given the timespan between matching GPT-3.5 (Mixtral perhaps?) and matching GPT-4 has just been a few weeks, I am wondering if the open source models have more momentum.

That said, I am very curious what OpenAI has in their labs... Are they actually barely ahead? Or do they have something much better that is not yet public? Perhaps they were waiting for Llama 3 to show it? Exciting times ahead either way!

Re: Meta Llama 3

#187

Post by momofuku »

Surprisingly, looks like Mark Zuckerberg is listed as a contributor in the Model Card [1]. I thought since its a pretty big effort, most executives would be added to it as well, but that does not seem to be the case at all. In fact I was surprised that Soumith Chintala was left out here [2].

[1] - https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md...

[2] - https://soumith.ch/about/

Re: Meta Llama 3

#188

Post by aster0id »

lordswork wrote: post #132
chaoz_ wrote: post #58

Earlier quoted context omitted.

I can't express how good Dwarkesh's podcast is in general.

Lex walked so that Dwarkesh could run. He runs the best AI podcast around right now, by a long shot.

I agree that it is the best AI podcast.

I do have a few gripes though, which might just be from personal preference. A lot of the time the language used by both the host and the guests is unnecessarily obtuse. Also the host is biased towards being optimistic about LLMs leading to AGI, and so he doesn't probe guests deep enough about that, more than just asking something along the lines of "Do you think next token prediction is enough for AGI?". Most of his guests are biased economically or academically to answer yes. This is then taken as the premise of the discussion following.

Having said that, I do agree that it is much better and deeper than other podcasts about AI.

Re: Meta Llama 3

#189

Post by gpm »

chown wrote: post #184

If anyone is looking to try 7B locally really quick, we have just added it to Msty. [1]: https://msty.app

From the faq

> Does Msty support GPUs?

> Yes on MacOS. On Windows* only Nvidia GPU cards are supported; AMD GPUs will be supported soon.

Do you support GPUs on linux?

Your downloads with windows are also annotated with CPU/CPU + GPU, but your linux ones aren't. Does that imply they are CPU only?

Re: Meta Llama 3

#190

Post by moffkalast »

alabhyajindal wrote: post #175

How does it make monetary sense to release open source models? AFAIK it's very expensive to train them. Do Meta/Mistral have any plans to monetize them?

Meta (iirc Zuck in some interview) has stated before that monetizing the models directly isn't part of their plan, they intend to profit from platforms that are built around them, e.g. targeting ads better by having a model look at people's messenger history. It's in their interest to have the community do free work to improve their architecture.

Mistral is monetizing them directly, they sell API access.

Post reply on HN