Live data from Hacker News

Meta Llama 3

llama.meta.com

271–280 of 965 posts

Re: Meta Llama 3

#271

I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…

The world at large seems to hate Zuck but it’s good to hear from people familiar with software engineering and who understand just how significant his contributions to open source and raising salaries have been through Facebook and now Meta.

> his contributions to ... raising salaries

It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That just concentrates the industry in fewer hands and makes it more dependent on fickle cash sources (investors, market expansion) often disconnected from the actual software being produced by their teams.

Nor is it great for the yet-to-mature craft that high salaries invited a very large pool of primarly-compensation-motivated people who end up diluting the ability for primarily-craft-motivated people to find and coordinate with each other in pursuit of higher quality work and more robust practices.

Re: Meta Llama 3

#272

I’m impressed by the benchmarks but really intrigued by the press release with the example prompt ~”Tell me some concerts I can go to on Saturday”. Clearly they are able to add their Meta data to context, but are they also crawling the web? Could this be a surface to exfiltrate Meta data in ways that scraping/ APIs cannot?

He said they were using Bing and Google on the Dwarkesh podcast.

Re: Meta Llama 3

#273

Earlier quoted context omitted.

For the still training 400B: Llama 3 GPT 4(Published) BBH 85.3 83.1 MMLU 86.1 86.4 DROP 83.5 80.9 GSM8K 94.1 92.0 MATH 57.8 52.9 HumEv 84.1 74.4 Although it should be noted that the API numbers were generally better than published numbers for GPT4. [1]: https://deepmind.google/technologies/gemini/

Hm, how much VRAM would this take to run?

My guess is around 256GiB but it depends on what level of quantization you are okay with. At full 16bit it will be massive, near 512GiB.

I figure we will see some Q4's that can probably fit on 4 4090s with CPU offloading.

Re: Meta Llama 3

#274
post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

And they even allow you to use it without logging in. Didnt expect that from Meta.

I had the same reaction, but when I saw the thumbs up and down icon, I realized this was a smart way to crowd source validation data.

Re: Meta Llama 3

#275
post #240

Earlier quoted context omitted.

You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.

Depends on your size threshhold. For anything beyond 100 bn in market cap certainly. There is some relatively large companies with a similar flair though, like Cohere and obviously Mistral.

Well, they're not AI companies, necessarily, or at least not only AI companies, but the big hardware firms tend to have engineers at the helm. That includes Nvidia, AMD, and Intel. (Counterpoint: Apple)

Re: Meta Llama 3

#277
post #267

I'm a big fan of various AI companies taking different approaches. OpenAI keeping it close to their hearts but have great developer apis. Meta and Mistral going open weights + open code. Anthropic and Claude doing their thing. Competition is a beautiful thing. I am half excited and half scared that AGI is our generation's space war. I hope we can solve the big human problems, instead of more scammy ads and videos. So…

My personal theory is that this is all because Zuckerberg has a rivalry with Elon Musk, who is an AI decelerationist (well, when it's convenient for him) and appears to believe in keeping AI in the control of the few. There was a spat between them a few years ago on Twitter where Musk said Zuckerberg had limited understanding of AI tech, after Zuckerberg called out AI doomerism as stupid.

Re: Meta Llama 3

#278

Earlier quoted context omitted.

The world at large seems to hate Zuck but it’s good to hear from people familiar with software engineering and who understand just how significant his contributions to open source and raising salaries have been through Facebook and now Meta.

> his contributions to ... raising salaries It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That just concentrates the industry in fewer hands and makes it more dependent on fickle cash sources (investors, market expansion) often disconnected from the actual software being produced by their teams.…

I am fine with large pool of greedy people trying their hand at programming. Some of them will stick and find meaning in work. Rest will wade out in downturn. Net positive.

Re: Meta Llama 3

#279

I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…

Why is Meta doing it though? This is an astronomical investment. What do they gain from it?

Re: Meta Llama 3

#280

Earlier quoted context omitted.

It's likely RAG / augmented with web data. Would be interested if local execution returned the same results.

It is. You can see a little "G" icon indicating that it searched the web with Google.

Interesting, mine has a different icon and used Bing
Post reply on HN