Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

601–610 of 695 posts

Re: The Llama 4 herd

#601

Post by brookst »

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

Yeah that sounds like “the sum total of all human knowledge and thinking leans left”. At what point is it no longer a “bias” and just an observation that “leans left” is aligned with human nature?

Re: The Llama 4 herd

#602

Post by darksaints »

concordDance wrote: post #498
hijodelsol wrote: post #488

Earlier quoted context omitted.

Maybe because that position is both scientifically and morally unsound and if held strongly will lead to dehumanization and hate, attributes we should prevent any LLM from having.

That particular debate is often a semantics debate, so it isn't in the domain of science at all. The main way I can think of off-hand to try and make it scientific is to ask about correlational clusters. And then you get way more than two genders, but you definitely get some clusters that contain both transwomen and men (e.g. if I hear a video game speed runner or open source software passion projecf maker using she/…

I have noticed certain groups where trans people are relatively over represented and group involvement more correlated with biological gender, but that’s not actually that interesting or meaningful in reality. Trans women having similar interests to men doesn’t make them men any more than me owning a gun makes me a Republican.

Re: The Llama 4 herd

#603

Post by CooCooCaCha »

kergonath wrote: post #501
CooCooCaCha wrote: post #474

Earlier quoted context omitted.

There is no rhetoric here, it’s just literal truth. There is no implication of equivalence or any statement about the value of objective truth. Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.

> Any position is a bias. A flat earther would consider a round-earther biased. That’s bollocks. The Earth is measurably not flat. You start from a position of moral relativism and then apply it to falsifiable propositions. It’s really not the same thing. Some ideas are provably false and saying that they are false is not "bias".

Dice are considered "biased" if not all sides have equal probability, even if that's literally true.

When you look up the definition of bias you see "prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair."

So the way we use the word has an implication of fairness to most people, and unfortunately reality isn't fair. Truth isn't fair. And that's what I'm trying to point out here in reference to LLM output.

Re: The Llama 4 herd

#604

Post by janwas »

halifaxbeard wrote: post #583
tucnak wrote: post #560

Earlier quoted context omitted.

If you really were serious about privacy, you wouldn't put yourself at disadvantage with a locked-down six-year out of date card. Tenstorrent Blackhole exists now, btw.

I think it's disingenuous to suggest they're putting themselves at a disadvantage with an RTX 3090, especially in a comparison to an inferior product that isn't even shipping yet. RTX 3090: 24GB RAM, 936.2GB/s bandwidth Tenstorrent p150a: 32GB RAM, 512GB/s bandwidth an extra 8GB of ram isn't worth nearly halving memory bandwidth.

Or how about https://www.notebookcheck.net/Way-to-run-DeepSeek-s-671B-AI-... 768 GiB for $6000.

Re: The Llama 4 herd

#605

Post by wrs »

jimmyl02 wrote: post #315
zamadatix wrote: post #297

Earlier quoted context omitted.

The only thing about this which may be unintuitive from the name is an "Expert" is not something like a sub-llm that's good at math and gets called when you ask a math question. Models like this have layers of networks they run tokens through and each layer is composed of 256 sub-networks, any of which can be selected (or multiple selected and merged in some way) for each layer independently. So the net result is the…

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

ML folks tend to invent fanciful metaphorical terms for things. Another example is “attention”. I’m expecting to see a paper “consciousness is all you need” where “consciousness” turns out to just be a Laplace transform or something.

Re: The Llama 4 herd

#606

Post by cpeterson42 »

For anyone looking to experiment with these models who doesn't have 210GB of VRAM on tap-we're working as quickly as we can to get cheap access to 4x80GB A100 instances running at thundercompute.com (aiming for sub-$5/hr). For quantized versions, we have cheaper 1-2 GPU nodes available today. If you're interested, join our Discord for updates: https://discord.com/invite/nwuETS9jJK

Re: The Llama 4 herd

#607

Post by [deleted] »

darksaints wrote: post #602
concordDance wrote: post #498

Earlier quoted context omitted.

That particular debate is often a semantics debate, so it isn't in the domain of science at all. The main way I can think of off-hand to try and make it scientific is to ask about correlational clusters. And then you get way more than two genders, but you definitely get some clusters that contain both transwomen and men (e.g. if I hear a video game speed runner or open source software passion projecf maker using she/…

I have noticed certain groups where trans people are relatively over represented and group involvement more correlated with biological gender, but that’s not actually that interesting or meaningful in reality. Trans women having similar interests to men doesn’t make them men any more than me owning a gun makes me a Republican.

[deleted]

Re: The Llama 4 herd

#608

Post by naenin »

darksaints wrote: post #602
concordDance wrote: post #498

Earlier quoted context omitted.

That particular debate is often a semantics debate, so it isn't in the domain of science at all. The main way I can think of off-hand to try and make it scientific is to ask about correlational clusters. And then you get way more than two genders, but you definitely get some clusters that contain both transwomen and men (e.g. if I hear a video game speed runner or open source software passion projecf maker using she/…

I have noticed certain groups where trans people are relatively over represented and group involvement more correlated with biological gender, but that’s not actually that interesting or meaningful in reality. Trans women having similar interests to men doesn’t make them men any more than me owning a gun makes me a Republican.

[flagged]

Re: The Llama 4 herd

#609

Post by kiitos »

martin82 wrote: post #479
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

No it is not. Right leaning opinions are heavily censored and shunned in all major publishing platforms that bots can scrape. For example, before Trump, if you contested the utterly normal common sense and scientifically sound idea that a trans woman is still a man, you would be banned - therefore, people with common sense will simply disengage, self-censor and get on with life.

Hate to break it to you, but gender is not an immutable/normative property defined forever at birth, it's a mutable/descriptive property evaluated in context. For example, in the year of our lord 2025, Hunter Schafer is a woman, with no ifs, ands, or buts.
Post reply on HN