Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

591–600 of 965 posts

Re: Meta Llama 3

#591

Post by eigenvalue »

a_wild_dandan wrote: post #573
skybrian wrote: post #546

Earlier quoted context omitted.

How does that work? Nobody will be able to run the big models who doesn't have a big data center or lots of rent money to burn. How is it going to matter to most of us? It seems similar to open chip designs - irrelevant to people who are going to buy whatever chips they use anyway. Maybe I'll design a circuit board, but no deeper than that. Modern civilization means depending on supply chains.

The day it's released, Llama-3-405B will be running on someone's Mac Studio. These models aren't that big. It'll be fine, just like Llama-2.

Maybe at 1 or 2 bits of quantization! Even the Macs with the most unified RAM are maxxed out with much smaller models than 405b (especially since it's a dense model and not a MOE).

Re: Meta Llama 3

#592

Post by buildbot »

Manabu-eo wrote: post #463
buildbot wrote: post #87

Quick thoughts - Major arch changes are not that major, mostly GQA and tokenizer improvements. Tokenizer improvement is a under-explored domain IMO. 15T tokens is a ton! 400B model performance looks great, can’t wait for that to be released. Might be time to invest in a Mac studio! OpenAI probably needs to release GPT-5 soon to convince people they are still staying ahead.

> Might be time to invest in a Mac studio! The highest end Mac Studio with 196GB of ram won't even be enough to run a Q4 quant of the 400B+ (don't forget the +) model. At this point, one have to consider an Epyc for CPU inference or costlier gpu solutions like the "popular" 8xA100 80GB... An if it's a dense model like the other llamas, it will be pretty slow..

It might be large enough Q2 won't impact performance too much (not guaranteed), but that's a great point.

Time to invest in a Milan-X! (Already have the 512GB of DDR4)

Re: Meta Llama 3

#593

Post by int_19h »

freedomben wrote: post #370
resource_waste wrote: post #55

Earlier quoted context omitted.

[flagged]

I haven't tried Llama 3 yet, but Llama 2 is indeed extremely "safe." (I'm old enough to remember when AI safety was about not having AI take over the world and kill all humans, not when it might offend a Puritan's sexual sensibilities or hurt somebody's feelings, so I hate using the word "safe" for it, but I can't think of a better word that others would understand). It's not quite as bad as Gemini, but in the same c…

"Censored" is the word that you're looking for, and is generally what you see when these models are discussed on Reddit etc.

Not to worry - uncensored finetunes will be coming shortly.

Re: Meta Llama 3

#594

Post by selcuka »

nathanh4903 wrote: post #582

I tried generating a Chinese rap song, and it did generate a pretty good rap. However, upon completion, it deleted the response, and showed > I don’t understand Chinese yet, but I’m working on it. I will send you a message when we can talk in Chinese. I tried some other languages and the same. It will generate non-English language, but once its done, the response is deleted and replaced with the message

I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been generated, and reverts the response if it's below a threshold.

Re: Meta Llama 3

#595

Post by a_wild_dandan »

eigenvalue wrote: post #438
a_wild_dandan wrote: post #425

Earlier quoted context omitted.

Also, being open source adds phenomenal value for Meta: 1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage. 2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible. 3. The industry standardizes around their tech, driving down costs and d…

Yes, I completely agree with every point you made. It’s going to be so satisfying when all the AI safety people realize that their attempts to cram this protectionist/alarmist control down our throats are all for nothing, because there is an even stronger model that is totally open weights, and you can never put the genie back in the bottle!

[flagged]

Re: Meta Llama 3

#596

Post by HarHarVeryFunny »

simonw wrote: post #204
jph00 wrote: post #160

Earlier quoted context omitted.

> " Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing. " You seem to be making claims that have little connection to the actual license. The license states yo…

Those additional restrictions mean it's not an open source license by the OSI definition, which matters if you care about words sometimes having unambiguous meanings. I call models like this "openly licensed" but not "open source licensed".

Call it what you will, but it'd be silly if Meta let these 700M+ customer mega-corps (Amazon, Google, etc) just take Meta models and sell access to them without sharing revenue with Meta.

You should be happy that Meta find ways to make money from their models, otherwise it's unlikely that they'd be giving you free access (until your startup reaches 700M+ customers, when the free ride ends).

Re: Meta Llama 3

#599

Post by int_19h »

modeless wrote: post #287
sp332 wrote: post #112

Earlier quoted context omitted.

The bottom of https://ai.meta.com/blog/meta-llama-3/ has in-progress results for the 400B model as well. Looks like it's not quite there yet. Llama 3 400B Base / Instruct MMLU 84.8 86.1 GPQA - 48.0 MATH - 57.8 HumanEval - 84.1 DROP 83.5 -

Not quite there yet, but very close and not done training! It's quite plausible that this model could be state of the art over GPT-4 in some domains when it finishes training, unless GPT-5 comes out first. Although 400B will be pretty much out of reach for any PC to run locally, it will still be exciting to have a GPT-4 level model in the open for research so people can try quantizing, pruning, distilling, and other…

The real question will be, how much you can quantize that while still retaining sanity. 400b at 2-bit would be possible to run on a Mac Studio - probably at multiple seconds per token, but sometimes that's "fast enough".

Re: Meta Llama 3

#600

Post by soulofmischief »

mensetmanusman wrote: post #589
littlestymaar wrote: post #486

Earlier quoted context omitted.

> It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read. Linux is open source, because you can actually compile it yourself! You don't need Linus's email for that (and if you needed some secret cryptographic key on Linus' laptop to decrypt and compile the kernel, then…

It's a blob that costs over $10,000,000 in electricity costs to compile. Even if they released everything only the rich could push go.

There is an argument to be made about the importance of archeological preservation of the provenance of models, especially the first few important LLMs, for study by future generations.

In general, software rot is a huge issue, and many projects which may be of future archeological importance are increasingly non-reproducible as dependencies are often not vendored and checked into source, but instead downloaded at compile time from servers which lack strong guarantees about future availability.

Post reply on HN