Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

321–330 of 695 posts

Re: The Llama 4 herd

#321

Post by averageRoyalty »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

40% of Americans is about 2% of the worlds population though.

It's hardly biased, it's stating the current scientific stance over a fringe belief with no evidence.

Re: The Llama 4 herd

#323

Post by TacticalCoder »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

[dead]

Re: The Llama 4 herd

#324

Post by mlyle »

TOMDM wrote: post #170
p12tic wrote: post #134

Earlier quoted context omitted.

For all intents and purposes cache may not exist when the working set is 17B or 109B parameters. So it's still better that less parameters are activated for each token. 17B parameters works ~6x faster than 109B parameters just because less data needs to be loaded from RAM.

Yes loaded from RAM and loaded to RAM are the big distinction here. It will still be slow if portions of the model need to be read from disk to memory each pass, but only having to execute portions of the model for each token is a huge speed improvement.

It's not too expensive of a Macbook to fit 109B 4-bit parameters in RAM.

Re: The Llama 4 herd

#326

Post by theGnuMe »

Cyphase wrote: post #310
senderista wrote: post #226

Earlier quoted context omitted.

I see leftists denying inconvenient facts just as much as rightists. It's just the inevitable product of a tribal mentality, the tribe doesn't matter.

https://www.paulgraham.com/mod.html > There are two distinct ways to be politically moderate: on purpose and by accident. Intentional moderates are trimmers, deliberately choosing a position mid-way between the extremes of right and left. Accidental moderates end up in the middle, on average, because they make up their own minds about each question, and the far right and far left are roughly equally wrong.

I never liked this answer. Moderates could just be wrong.

Re: The Llama 4 herd

#327

Post by tomp »

jimmyl02 wrote: post #315
zamadatix wrote: post #297

Earlier quoted context omitted.

The only thing about this which may be unintuitive from the name is an "Expert" is not something like a sub-llm that's good at math and gets called when you ask a math question. Models like this have layers of networks they run tokens through and each layer is composed of 256 sub-networks, any of which can be selected (or multiple selected and merged in some way) for each layer independently. So the net result is the…

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

> individual tokens are routed to different experts

that was AFAIK (not an expert! lol) the traditional approach

but judging by the chart on LLaMa4 blog post, now they're interleaving MoE models and dense Attention layers; so I guess this means that even a single token could be routed through different experts at every single MoE layer!

Re: The Llama 4 herd

#328

Post by mdp2021 »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

> If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old

It will have to reply "According to Clair Patterson and further research, the Earth is ~4.5 billion years old". Or some other form that points to the source somewhere.

Re: The Llama 4 herd

#329

Post by kristopolous »

littlestymaar wrote: post #314
taneq wrote: post #308

Earlier quoted context omitted.

Unless something’s changed you will need the whole model on the HPU anyway, no? So way beyond a 4090 regardless.

You can still offload most of the model to RAM and use the GPU for compute, but it's obviously much slower than what it would be if everything was on the GPU memory. see ktransformers: https://www.reddit.com/r/LocalLLaMA/comments/1jpi0n9/ktransf...

I'm certainly not the brightest person in this thread but has there been effort to maybe bucket the computational cost of the model so that more expensive parts are on the gpu and less expensive parts are on the cpu?

Re: The Llama 4 herd

#330

Post by paxys »

Rover222 wrote: post #262
paxys wrote: post #230

Earlier quoted context omitted.

That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

So google Gemini was creating black Vikings because of facts?

Should an "unbiased" model not create vikings of every color? Why offend any side?
Post reply on HN