Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

401–410 of 695 posts

Re: The Llama 4 herd

#401

Post by EasyMark »

kubb wrote: post #210
hannasanarion wrote: post #101

Earlier quoted context omitted.

Or it is more logically and ethically consistent and thus preferable to the models' baked in preferences for correctness and nonhypocrisy. (democracy and equality are good for everyone everywhere except when you're at work in which case you will beg to be treated like a feudal serf or else die on the street without shelter or healthcare, doubly so if you're a woman or a racial minority, and that's how the world shoul…

LLMs are great at cutting through a lot of right (and left) wing rhetorical nonsense. Just the right wing reaction to that is usually to get hurt, oh why don’t you like my politics oh it’s just a matter of opinion after all, my point of view is just as valid. Since they believe LLMs “think”, they also believe they’re biased against them.

I think right wing tends to be much less "tolerant" of live and let live, as religions are often a huge part of their "bias" and those religions often say that others must be punished for not following God's(s') path, up and including destruction of those who don't fall in line.

Re: The Llama 4 herd

#402

Post by simonw »

kristianp wrote: post #395

I'd like to discuss the matter of size. Llama has gone from talking up an 8b model as capable to having a smallest model of 109b. What will be the sizes in a years time? Things are moving out of reach for commodity pc's, 128GB is possible, but expensive.

I'm hoping that Llama 4 goes the same way as Llama 3.

The first Llama 3 models released were 8B and 70B in April 2024.

Llama 3.1 came later in July at 8B, 70B, and 405B.

Llama 3.2 in September got really interesting: 1B, 3B, 11B and 90B.

Then Llama 3.3 in December was 70B but claimed performance similar to the earlier Llama 3.1 405B!

Llama 4 is 109B and 400B, both of which were trained with the help of the 2T(?) "Behemoth".

I'm hoping we'll see further releases in the Llama 4 series that are smaller. I'm particularly excited to see if they produce a ~24B model, since that appears to be the sweet spot for running models on my 64GB laptop while still being able to have other applications running at the same time. Mistral Small 3.1 is a 24B model and is absolutely superb.

(Fleshed this comment out a bit on my blog: https://simonwillison.net/2025/Apr/5/llama-4-notes/#my-hopes...)

Re: The Llama 4 herd

#403

Post by nickysielicki »

tucnak wrote: post #367
hydroreadsstuff wrote: post #243

This means GPUs are dead for local enthusiast AI. And SoCs with big RAM are in. Because 17B active parameters should reach enough performance on 256bit LPDDR5x.

This has been the case for a while now. 3090 hoarders were always just doing it for street cred or whatever, no way these guys are computing anything of actual value. Tenstorrent is on fire, though. For small businesses this is what matters. If 10M context is not a scam, I think we'll see SmartNIC adoption real soon. I would literally long AMD now because their Xilinx people are probably going to own the space real s…

> Infiniband is cool and all, but it's also stupid and their scale-out strategy is non-existent.

god I love this website.

Re: The Llama 4 herd

#405

Post by econ »

EasyMark wrote: post #400
tensor wrote: post #353

Earlier quoted context omitted.

Call me crazy, but I don't want an AI that bases its reasoning on politics. I want one that is primarily scientific driven, and if I ask it political questions it should give me representative answers. E.g. "The majority view in [country] is [blah] with the minority view being [bleh]." I have no interest in "all sides are equal" answers because I don't believe all information is equally informative nor equally true.

But if you don't incorporate some moral guidelines, I think if an AI is left to strictly decide what is best to happen to humans it will logically conclude that there needs to be a lot less of us or none of us left, without some bias tossed in there for humanistic concerns. The universe doesn't "care" if humans exist or not, but our impact on the planet is a huge negative if one creature's existence is as important a…

The size of their brain may depend on how many people are in the economy.

Re: The Llama 4 herd

#406

Post by fsndz »

laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

Nice release. I see that everyone is playing the differentiation game now: https://medium.com/thoughts-on-machine-learning/llama-4-and-...

Re: The Llama 4 herd

#407

Post by highfrequency »

Crazy that there are now five and a half companies that all have roughly state of the art LLMs.

> We developed a new training technique which we refer to as MetaP that allows us to reliably set critical model hyper-parameters such as per-layer learning rates and initialization scales. We found that chosen hyper-parameters transfer well across different values of batch size, model width, depth, and training tokens.

This sounds interesting. Anyone have a link to the paper or other documentation on MetaP?

Re: The Llama 4 herd

#408

Post by aurareturn »

elorant wrote: post #356
echelon wrote: post #116

Earlier quoted context omitted.

I don't understand Framework's desktop offerings. For laptops their open approach makes sense, but desktops are already about as hackable and DIY as they come.

It’s an x86 PC with unified RAM based on AMD’s new AI cpus. Pretty unique offering. Similar to Mac studio but you can run Linux or Windows on it, and it’s cheaper too.

It's a lot slower than a Mac Studio. Significantly slower CPU, GPU, memory bandwidth.

Re: The Llama 4 herd

#409

Post by phire »

jimmyl02 wrote: post #315
zamadatix wrote: post #297

Earlier quoted context omitted.

The only thing about this which may be unintuitive from the name is an "Expert" is not something like a sub-llm that's good at math and gets called when you ask a math question. Models like this have layers of networks they run tokens through and each layer is composed of 256 sub-networks, any of which can be selected (or multiple selected and merged in some way) for each layer independently. So the net result is the…

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

It's not even per token. The routing happens once per layer, with the same token bouncing between layers.

It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant).

It's a performance optimization for large deployments with thousands of GPUs answering tens of thousands of queries per second. They put thousands of queries into a single batch and run them in parallel. After each layer, the queries are re-routed to the GPU holding the correct subset of weights. Individual queries will bounce across dozens of GPUs per token, distributing load.

Even though the name "expert" implies they should experts in a given topic, it's really not true. During training, they optimize for making the load distribute evenly, nothing else.

Re: The Llama 4 herd

#410

Post by 1024core »

Anyone know what they mean by this:

> We developed a novel distillation loss function that dynamically weights the soft and hard targets through training.

Post reply on HN