Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

331–340 of 695 posts

Re: The Llama 4 herd

#331

Post by jhugg »

clueless wrote: post #128
laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

> Knowledge cutoff: August 2024. Could this mean training time is generally around 6 month, with 2 month of Q/A?

I wish my knowledge cutoff was August 2024.

Re: The Llama 4 herd

#332

Post by senderista »

theGnuMe wrote: post #326
Cyphase wrote: post #310

Earlier quoted context omitted.

https://www.paulgraham.com/mod.html > There are two distinct ways to be politically moderate: on purpose and by accident. Intentional moderates are trimmers, deliberately choosing a position mid-way between the extremes of right and left. Accidental moderates end up in the middle, on average, because they make up their own minds about each question, and the far right and far left are roughly equally wrong.

I never liked this answer. Moderates could just be wrong.

"Intentional moderate" is certainly just another tribe. Aiming squarely for the middle of the Overton window du jour is sort of a politician's job, but it shouldn't be emulated by others.

Re: The Llama 4 herd

#333

Post by shreezus »

Haven't had a chance to play with this yet, but 10M context window is seriously impressive. I think we'll see models with 100M context relatively soon, and eliminate the need for RAG for a lot of use cases.

Re: The Llama 4 herd

#335

Post by vessenes »

Buttons840 wrote: post #296
vessenes wrote: post #289

Earlier quoted context omitted.

This was an idea that sounded somewhat silly until it was shown it worked. The idea is that you encourage through training a bunch of “experts” to diversify and “get good” at different things. These experts are say 1/10 to 1/100 of your model size if it were a dense model. So you pack them all up into one model, and you add a layer or a few layers that have the job of picking which small expert model is best for your…

If I have 5000 documents about A, and 5000 documents about B, do we know whether it's better to train one large model on all 10,000 documents, or to train 2 different specialist models and then combine them as you describe?

well you don't. but the power of gradient descent if properly managed will split them up for you. But you might get more mileage out of like 200 specialist models.

Re: The Llama 4 herd

#336

Post by kristianp »

terhechte wrote: post #26

The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked a local 7B model (qwen 2.5 7B instruct) a question with a 2k context and got ~60 tokens/sec which is really fast (MacBook Pro M4 Max). So this could hit 30 token/sec. Time to first…

To clarify, you're still gonna want enough RAM for the entire model plus context. Scout being 109B params means 64GB at q4, but then your context and other applications will have about 9GB left to work with.

Re: The Llama 4 herd

#337

Post by kristianp »

randunel wrote: post #222
mdp2021 wrote: post #197

Earlier quoted context omitted.

And given that some people are afraid of malicious software in some brands of mini-PCs on the market, to have some more trusted product around will also be an asset.

Lenovo backdoors as preinstalled software, including their own TLS certificate authorities. Name whom you're referring to every time!

Is that still a thing?

Re: The Llama 4 herd

#338

Post by vessenes »

LeafItAlone wrote: post #291
vessenes wrote: post #275

Earlier quoted context omitted.

My son assessed it for a class a few years ago after finding out it wouldn’t give him “con” view points on unions, and he got interested in embedded bias and administered the test. I don’t have any of the outputs from the conversation, sadly. But replication could be good! I just fired up GPT-4 as old as I could get and checked; it was willing to tell me why unions are bad, but only when it could warn me multiple tim…

I hope on HN that we hold ourselves to a higher standard for “it’s been true from the beginning” than a vague recall of “My son assessed it for a class a few years ago” and not being able to reproduce.

I literally went back to the oldest model I could access and hand verified that in fact it does what I described, which is lecture you if you don't like unions and goes sweetly along if you do like unions. I feel this is a fair and reasonably well researched existence proof for a Saturday afternoon, and propose that it might be on you to find counter examples.
Post reply on HN