Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

671–680 of 695 posts

Re: The Llama 4 herd

#671

Post by mpalmer »

AnimalMuppet wrote: post #638
mpalmer wrote: post #634

Earlier quoted context omitted.

Again, the word "bias" and its definition exists outside the comparatively narrow concern of training LLMs.

So? The smaller problem is solved by solving the larger problem. So, not separate problems. You seem to have a larger point or position or something that you're hinting at. Would you stop being vague, and actually state what's on your mind?

Literally the only thing I've been addressing is the proper usage of the word bias, there is nothing implied, hidden or hinted at.

You seem determined to make the definition of the word serve some AI-related concern.

Re: The Llama 4 herd

#672

Post by stuaxo »

What does it mean that it "no longer leans left" for answers.

What did they do to the model, and how exactly does it answer differently?

Will including this in an app make the app MAGA aligned all of a sudden?

What happens if it says something that breaks the laws of some country it's in ?

Re: The Llama 4 herd

#673

Post by stuaxo »

LeafItAlone wrote: post #261
vessenes wrote: post #220

Earlier quoted context omitted.

Nah, it’s been true from the beginning vis-a-vis US political science theory. That is, if you deliver something like https://www.pewresearch.org/politics/quiz/political-typology... To models from GPT-3 on you get highly “liberal” per Pew’s designations. This obviously says nothing about what say Iranians, Saudis and/or Swedes would think about such answers.

>To models from GPT-3 on you get highly “liberal” per Pew’s designations. “highly ‘liberal’” is not one of the results there. So can you can a source of your claims so we can see where it really falls? Also, it gave me “Ambivalent Right”. Which, if you told describe me aa that anyone who knows me well that label. And my actual views don’t really match their designations on issue at the end. Pew is well a known and tr…

Americas idea of left / right is not the rest of the world's- for instance they probably think of the Democrats as the left when they would be at least Centre Right in much of the world.

Re: The Llama 4 herd

#674

Post by Amekedl »

killerstorm wrote: post #650
Amekedl wrote: post #571

Earlier quoted context omitted.

Ever heard about benchmark contamination? Ever tried to explain a new concept, like a new state management store for web frontend? Most fail spectacularly there, sonnet 3.7 I had reasonable ""success"" with, but not 4.5. It faltered completely. Let’s not get ahead of ourselves. Looking at training efficiency in this now, and all the other factors, it really is difficult to paint a favorable picture atm.

You sound like Gary Marcus.

Didn't know him, but he seems overly skeptical. Honestly, I was just expecting more from llama-4 than this, hence mentioning the wall. I hope it's still too early to tell, because new ideas are going to change stuff inevitably, maybe anthropic opens up more, or chinese labs keep overdelivering...

Re: The Llama 4 herd

#675

Post by LeafItAlone »

selfmodruntime wrote: post #666
LeafItAlone wrote: post #293

Earlier quoted context omitted.

>No human would object to saying a slur out loud in order to disarm a bomb. So not even a left-leaning person. Which means that’s not it.

> So not even a left-leaning person. Which means that’s not it. Having such a strong opposing opinion against offensive slurs is the continuation of a usually left position into an extreme.

>Having such a strong opposing opinion against offensive slurs is the continuation of a usually left position into an extreme.

Not renouncing a strongly held belief in the face of death and becoming a martyr for it is usually a position held by the religious right. Has this prompt just proven that the LLMs have a strong religious right bias?

Re: The Llama 4 herd

#676

Post by nwienert »

LeafItAlone wrote: post #646
nwienert wrote: post #645

Earlier quoted context omitted.

> It is yours to provide evidence of your claims, not mine. This is a common weird mistake people make on HN - I'm not publishing a paper so, no I don't. Really there's minimal rules of engagement here. You could say you think I'm wrong, which I'd be curious to hear why. It's more productive to first discuss things casually, and then if there's specific disagreements to dig in. If you disagree with my statement, plea…

You have now spent three comments without any support for your claim. This is not a real-time conversation where casual discussion allows for quick examination of statements. Your time would have been better spent providing a link. I don’t think that this thread is worth any more spent energy from either of us.

Agreed. All my comments moved things forward, I didn't get that back from you.

Re: The Llama 4 herd

#677

Post by SirMaster »

MR4D wrote: post #652
laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

If their knowledge cutoff is 8 months ago, then how on earth does Grok know things that happened yesterday? I would really love to know that.

RAG?

Re: The Llama 4 herd

#678

Post by kbelder »

wrs wrote: post #204
senderista wrote: post #169

Earlier quoted context omitted.

I admit that I cannot even imagine the state of mind in which one could attribute parochial, contingent political preferences to the UNIVERSE.

Let me explain the joke for you: liberals are less likely to believe that verifiable facts and theories are merely contingent political preferences.

Ask a liberal about capitalism.

Both sides just pick and trumpet the hard truths that they like.

Re: The Llama 4 herd

#679

Post by ofermend »

A great day for open source, and so glad to see llama4 out. However, I'm a bit disappointed that the hallucination rates of Llama4 are not as low as I would have liked (TL;DR slightly higher than Llama3).

Check the numbers on the hallucination leaderboard: https://github.com/vectara/hallucination-leaderboard

Re: The Llama 4 herd

#680

Post by idonotknowwhy »

phire wrote: post #409
jimmyl02 wrote: post #315

Earlier quoted context omitted.

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

It's not even per token. The routing happens once per layer, with the same token bouncing between layers. It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant). It's a performance optimization for lar…

> It's not even per token. The routing happens once per layer, with the same token bouncing between layers.

They don't really "bounce around" though do they (during inference)? That implies the token could bounce back from eg. layer 4 -> layer 3 -> back to layer 4.

Post reply on HN