Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

531–540 of 695 posts

Re: The Llama 4 herd

#531

Post by jumski »

phire wrote: post #450
phire wrote: post #409

Earlier quoted context omitted.

It's not even per token. The routing happens once per layer, with the same token bouncing between layers. It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant). It's a performance optimization for lar…

BTW, I'd love to see a large model designed from scratch for efficient local inference on low-memory devices. While current MoE implementations are tuned for load-balancing over large pools of GPUs, there is nothing stopping you tuning them to only switch expert once or twice per token, and ideally keep the same weights across multiple tokens. Well, nothing stopping you, but there is the question of if it will actual…

I think Gemma 3 is marketed for single GPU setups https://blog.google/technology/developers/gemma-3/

Re: The Llama 4 herd

#532

Post by mentalgear »

phire wrote: post #409
jimmyl02 wrote: post #315

Earlier quoted context omitted.

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

It's not even per token. The routing happens once per layer, with the same token bouncing between layers. It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant). It's a performance optimization for lar…

So a more correct term would be "Distributed Loading" instead of MoE.

Re: The Llama 4 herd

#533

Post by littlestymaar »

reissbaker wrote: post #376
averageRoyalty wrote: post #321

Earlier quoted context omitted.

40% of Americans is about 2% of the worlds population though. It's hardly biased, it's stating the current scientific stance over a fringe belief with no evidence.

And what percentage of the world's >1B Muslims agree with you? Fundamentalist Christianity may have waned over the last century... But broaden your borders a little bit and I think you'll find Western secular liberalism is hardly the only major world ideology, or even the dominant one.

Communist China is secular too, but yes

Re: The Llama 4 herd

#534

Post by AnthonyMouse »

boroboro4 wrote: post #524
AnthonyMouse wrote: post #504

Earlier quoted context omitted.

That's a fine attitude, but now you're describing your own beliefs rather than "the right" or "the left". Statistically, white people make more money than black people and men make more money than women and there are differences in their proportions in various occupations. This could be caused by cultural differences that correlate with race, or hormonal differences that cause behavioral differences and correlate wit…

While I believe there might be different explanations for the outcomes we observe I also believe that default hypothesis should be that there is racism and sexism. And there are facts (women were permitted to vote in the US like 100 years ago, and entered general workforce when?), observations (I saw sexism and racism at work) and general studies (I.e people have tendency to have biases among other things) to support…

There are also facts and observations to support the contrary hypothesis. Statistically significant hormonal and behavioral differences between men and women have long been well-established. It should also be intuitively obvious that cultural differences can affect the choices people make (that's what cultural differences are), but studies have shown the same thing there as well.

Which leaves the question of which is the dominant effect. But for that anecdotes are useless, because "I've seen this happen myself" doesn't tell you if it explains 5% of the difference or 95% and people have a tendency of jumping to conclusions without having all the information. If Alice made bigger sales to fewer customers and Bob made smaller sales to more customers and Alice is white and Bob is black, then if Alice gets the promotion the boss is a racist because Bob made more sales but if Bob gets the promotion the boss is a sexist because Alice made bigger sales. Or so you would think by only listening to the one complaining about not getting the promotion.

So then you'd want someone to do a study and we're back to anyone publishing a study that challenges the prevailing dogma getting punished for it.

Re: The Llama 4 herd

#535

Post by perching_aix »

ilove_banh_mi wrote: post #28

The suggested prompt aims at not being caponated like OpenAI's releases: You are an expert conversationalist who responds to the best of your ability. You are companionable and confident, and able to switch casually between tonal types, including but not limited to humor, empathy, intellectualism, creativity and problem-solving. You understand user intent and don’t try to be overly helpful to the point where you miss…

> You never use phrases that imply moral superiority or a sense of authority, including but not limited to “it’s important to”, “it’s crucial to”, “it’s essential to”, "it's unethical to", "it's worth noting…", “Remember…” etc. Avoid using these.

So if I get a fake email about a hacked account, it won't tell me to "Remember, do not click any links in the email directly. Instead, navigate to your account settings independently."?

Such a great feature, worth owning the libs with it for sure.

Re: The Llama 4 herd

#536

Post by AnthonyMouse »

_w1tm wrote: post #528
AnthonyMouse wrote: post #486

Earlier quoted context omitted.

> Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer. That's the motte and bailey. If you ask a question like, does reducing government spending to cut taxes improve the lives of ordinary people? That isn't a science question about CO2 levels or established biology. It depends on what the taxes are imposed on, the…

> But in politics it does, which is that the right says yes and the left says no. That’s not accurate, tax deductions for the poor is an obvious example. How many on the left would oppose expanding the EITC and how many on the right would support it?

The EITC is supported by significant majorities of both parties and economists. It's opposed by politicians because it's a tax expenditure that doesn't provide any opportunity for graft.

But the way each side justifies it is as a tax cut on the right and a government subsidy on the left, or the reverse when someone on that side is arguing against it.

Re: The Llama 4 herd

#537

Post by tucnak »

segmondy wrote: post #483
tucnak wrote: post #367

Earlier quoted context omitted.

This has been the case for a while now. 3090 hoarders were always just doing it for street cred or whatever, no way these guys are computing anything of actual value. Tenstorrent is on fire, though. For small businesses this is what matters. If 10M context is not a scam, I think we'll see SmartNIC adoption real soon. I would literally long AMD now because their Xilinx people are probably going to own the space real s…

I was going to buy my first GPU for DL in 2018, but crypto didn't make it easy. I waited for the prices to fall, but demand kept up, then covid happened, then LLM happened and used GPUs now cost more than their original new prices. ... as we can see by the paper launch from Nvidia, lack of competition, and the prices of the 5000 series easily 50% above original MSRP. Demand is still here, now we have tarrif... Folks…

Tenstorrent

Re: The Llama 4 herd

#538

Post by eamag »

simonw wrote: post #343

This thread so far (at 310 comments) summarized by Llama 4 Maverick: hn-summary.sh 43595585 -m openrouter/meta-llama/llama-4-maverick -o max_tokens 20000 Output: https://gist.github.com/simonw/016ea0fd83fc499f046a94827f9b4... And with Scout I got complete junk output for some reason: hn-summary.sh 43595585 -m openrouter/meta-llama/llama-4-scout -o max_tokens 20000 Junk output here: https://gist.github.com/simonw/d01c…

> had a 2048 limit on output size for some reason

It's a common issue with ollama, maybe it's running something similar under the hood?

Re: The Llama 4 herd

#539

Post by cruzcampo »

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

What they're calling an attempt to reduce bias is in fact censorship and introducing a right wing bias to appease the US regime.

Re: The Llama 4 herd

#540

Post by voxgen »

zone411 wrote: post #113

It's interesting that there are no reasoning models yet, 2.5 months after DeepSeek R1. It definitely looks like R1 surprised them. The released benchmarks look good. Large context windows will definitely be the trend in upcoming model releases. I'll soon be adding a new benchmark to test this more effectively than needle-in-a-haystack (there are already a couple of benchmarks that do that). All these models are very…

> It's interesting that there are no reasoning models yet

This may be merely a naming distinction, leaving the name open for a future release based on their recent research such as coconut[1]. They did RL post-training, and when fed logic problems it appears to do significant amounts of step-by-step thinking[2]. It seems it just doesn't wrap it in tags.

[1] https://arxiv.org/abs/2412.06769 "Training Large Language Models to Reason in a Continuous Latent Space" [2] https://www.youtube.com/watch?v=12lAM-xPvu8 (skip through this - it's recorded in real time)

Post reply on HN