The Llama 4 herd • Hacker News Board

Re: The Llama 4 herd

#501

Post by kergonath » Sun, Apr 06, 2025, 7:08 AM UTC

CooCooCaCha wrote: post #474

fourside wrote: post #273

Earlier quoted context omitted.

I’ve seen more of this type of rhetoric online in the last few years and find it very insidious. It subtly erodes the value of objective truth and tries to paint it as only one of many interpretations or beliefs, which is nothing more than a false equivalence. The concept of being unbiased has been around for a long time, and we’re not going to throw it away just because a few people disagree with the premise.

There is no rhetoric here, it’s just literal truth. There is no implication of equivalence or any statement about the value of objective truth. Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.

> Any position is a bias. A flat earther would consider a round-earther biased.

That’s bollocks. The Earth is measurably not flat.

You start from a position of moral relativism and then apply it to falsifiable propositions. It’s really not the same thing. Some ideas are provably false and saying that they are false is not "bias".

Re: The Llama 4 herd

#502

Post by Auracle » Sun, Apr 06, 2025, 7:12 AM UTC

hijodelsol wrote: post #488

martin82 wrote: post #479

Earlier quoted context omitted.

No it is not. Right leaning opinions are heavily censored and shunned in all major publishing platforms that bots can scrape. For example, before Trump, if you contested the utterly normal common sense and scientifically sound idea that a trans woman is still a man, you would be banned - therefore, people with common sense will simply disengage, self-censor and get on with life.

Maybe because that position is both scientifically and morally unsound and if held strongly will lead to dehumanization and hate, attributes we should prevent any LLM from having.

Well, you proved his point soundly. Imagine the downvotes he’d get on Reddit from people of opinions like your own.

Really? It’s scientifically unsound? Come on now.

Re: The Llama 4 herd

#503

Post by vintermann » Sun, Apr 06, 2025, 7:12 AM UTC

pornel wrote: post #299

InvOfSmallC wrote: post #266

Earlier quoted context omitted.

For a super ignorant person: Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each Those experts are LLM trained on specific tasks or what?

No, it's more like sharding of parameters. There's no understandable distinction between the experts.

I understand they're only optimizing for load distribution, but have people been trying to disentangle what the the various experts learn?

Re: The Llama 4 herd

#504

Post by AnthonyMouse » Sun, Apr 06, 2025, 7:14 AM UTC

EasyMark wrote: post #436

simplify wrote: post #419

Earlier quoted context omitted.

Everyone has a "religion" – i.e. a system of values they subscribe to. Secular Americans are annoying because they believe they don't have one, and instead think they're just "good people", calling those who break their core values "bad people".

I follow a secular humanist moral system as best I can. I have tolerance for those who have tolerance for me. I grew up amongst fundamentalist christians and fundamentalist anything (christian, muslim, buddhist, whatever) leave a bad taste in my mouth. I don't care about your religion just don't try to force it on me or try to make me live by its moral system and you won't hear a peep out of me about what you're doin…

That's a fine attitude, but now you're describing your own beliefs rather than "the right" or "the left".

Statistically, white people make more money than black people and men make more money than women and there are differences in their proportions in various occupations. This could be caused by cultural differences that correlate with race, or hormonal differences that cause behavioral differences and correlate with sex, or it could be caused by racism and sexism. Much of the left takes it as an effectively religious position that the latter predominates even into present day. Many of them are quite militant and aggressive about it, and in particular will try to ruin anyone who presents evidence to the contrary or who opposes policies that would actively perpetrate injustice if their sacred assumptions weren't true anymore. Which isn't consistent with "live and let live".

And that's the nature of politics. You're never passing a law by a margin of 53 to 47 because everybody agrees with it. That's the 53% telling the 47% how to live.

"Only the other side does this" is false purity. There are no saints in Washington.

Re: The Llama 4 herd

#505

Post by KingMob » Sun, Apr 06, 2025, 7:16 AM UTC

CooCooCaCha wrote: post #474

fourside wrote: post #273

Earlier quoted context omitted.

I’ve seen more of this type of rhetoric online in the last few years and find it very insidious. It subtly erodes the value of objective truth and tries to paint it as only one of many interpretations or beliefs, which is nothing more than a false equivalence. The concept of being unbiased has been around for a long time, and we’re not going to throw it away just because a few people disagree with the premise.

There is no rhetoric here, it’s just literal truth. There is no implication of equivalence or any statement about the value of objective truth. Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.

> truth itself is a bias

Ehh, bias connotes unfairness, but espousing the truth should be considered the fairest position.

In statistics, bias literally refers to an inaccurate distortion of results.

I get what you're trying to say, but I don't think it's a useful definition of bias.

Re: The Llama 4 herd

#506

Post by lostmsu » Sun, Apr 06, 2025, 7:20 AM UTC

terhechte wrote: post #55

scosman wrote: post #37

Earlier quoted context omitted.

At 109b params you’ll need a ton of memory. We’ll have to wait for evals of the quants to know how much.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

At 4 bit quant (requires 64GB) the price of Mac (4.2K) is almost exactly the same as 2x5090 (provided we will see them in stock). But 2x5090 have 6x memory bandwidth and probably close to 50x matmul compute at int4.

Re: The Llama 4 herd

#507

Post by KingMob » Sun, Apr 06, 2025, 7:21 AM UTC

senderista wrote: post #226

wrs wrote: post #204

Earlier quoted context omitted.

Let me explain the joke for you: liberals are less likely to believe that verifiable facts and theories are merely contingent political preferences.

I see leftists denying inconvenient facts just as much as rightists. It's just the inevitable product of a tribal mentality, the tribe doesn't matter.

Which facts? Please be specific.

Re: The Llama 4 herd

#508

Post by ianks » Sun, Apr 06, 2025, 7:24 AM UTC

Are we going to find out that Meta pirated libgen again, with zero recognition to the authors?

“Open-sourcing it” doesn’t magically absolve you of the irreparable damages you’ve caused society. You stole their life’s work so your company could profit off of rage-slop.

Re: The Llama 4 herd

#509

Post by kergonath » Sun, Apr 06, 2025, 7:27 AM UTC

simplify wrote: post #419

EasyMark wrote: post #401

Earlier quoted context omitted.

I think right wing tends to be much less "tolerant" of live and let live, as religions are often a huge part of their "bias" and those religions often say that others must be punished for not following God's(s') path, up and including destruction of those who don't fall in line.

Everyone has a "religion" – i.e. a system of values they subscribe to. Secular Americans are annoying because they believe they don't have one, and instead think they're just "good people", calling those who break their core values "bad people".

> Any position is a bias. A flat earther would consider a round-earther biased.

That is not what a religion is.

> Secular Americans are annoying because they believe they don't have one

Why is that a problem to you?

> and instead think they're just "good people", calling those who break their core values "bad people".

No, not really. Someone is not good or bad because you agree with them. Even a religious person can recognise that an atheist doing charitable work is being good, regardless of whether they share a specific set of belief.

The attitude you describe is wrong, and from my experience much more common in religious fundamentalists than radical atheists (the vast majority of people in western democracies do not care whether you have a religion). I have never seen an atheist saying that. But I’ve had priests telling me that I had not "rejected Satan" because I was not baptised.

Re: The Llama 4 herd

#510

Post by vintermann » Sun, Apr 06, 2025, 7:27 AM UTC

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I think this is just a loyalty statement, to be honest. Just like when a large corporation pretended to care a lot about pronouns, they didn't actually, they just wanted to flag allegiance to a certain interest coalition/patronage network.

And those people, for the most part, didn't really care much about pronouns either. And they knew no one else really did either. It was an ideological shibboleth to them, a safe and easy commitment since it affects so few people, and is unlikely to matter for anything they do care about.

Now Meta is shopping around for new markers. "Liberal bias" is a classic, that's still popular with the Trump-right. I don't think they mean much by that either.