Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

581–590 of 695 posts

Re: The Llama 4 herd

#581

Post by igravious »

phire wrote: post #409
jimmyl02 wrote: post #315

Earlier quoted context omitted.

the most unintuitive part is that from my understanding, individual tokens are routed to different experts. this is hard to comprehend with "experts" as that means two you can have different experts for two sequential tokens right? I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token . that concept feels difficult…

It's not even per token. The routing happens once per layer, with the same token bouncing between layers. It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant). It's a performance optimization for lar…

> making the load distribute evenly, nothing else.

so you mean a "load balancer" for neural nets … well, why don't they call it that then?

Re: The Llama 4 herd

#582

Post by eric_cc »

yieldcrv wrote: post #301
yojo wrote: post #259

Earlier quoted context omitted.

In comedy, they call this “punching down” vs “punching up.” If you poke fun at a lower status/power group, you’re hitting someone from a position of power. It’s more akin to bullying, and feels “meaner”, for lack of a better word. Ripping on the hegemony is different. They should be able to take it, and can certainly fight back. It’s reasonable to debate the appropriateness of emulating this in a trained model, thoug…

not everything an LLM is prompted for is comedy additionally, infantilizing entire groups of people is an ongoing criticism of the left by many groups of minorities, women, and the right. which is what you did by assuming it is “punching down”. the beneficiaries/subjects/victims of this infantilizing have said its not more productive than what overt racists/bigots do, and the left chooses to avoid any introspection o…

The leftist coddling crusades are just a different form of dominance over minorities. It absolutely is bigotry and sense of superiority driving it. That said, it would take one incredible therapist to get them to realize it.

Re: The Llama 4 herd

#583

Post by halifaxbeard »

tucnak wrote: post #560
xigency wrote: post #392

Earlier quoted context omitted.

Not a hoarder per-se but I bought a 24GB card on the secondary market. My privacy is valuable. I'm okay being a half-step or full-step behind in LLM or image diffusion if it means my data never leaves my machine.

If you really were serious about privacy, you wouldn't put yourself at disadvantage with a locked-down six-year out of date card. Tenstorrent Blackhole exists now, btw.

I think it's disingenuous to suggest they're putting themselves at a disadvantage with an RTX 3090, especially in a comparison to an inferior product that isn't even shipping yet.

RTX 3090: 24GB RAM, 936.2GB/s bandwidth

Tenstorrent p150a: 32GB RAM, 512GB/s bandwidth

an extra 8GB of ram isn't worth nearly halving memory bandwidth.

Re: The Llama 4 herd

#584

Post by Philpax »

tucnak wrote: post #560
xigency wrote: post #392

Earlier quoted context omitted.

Not a hoarder per-se but I bought a 24GB card on the secondary market. My privacy is valuable. I'm okay being a half-step or full-step behind in LLM or image diffusion if it means my data never leaves my machine.

If you really were serious about privacy, you wouldn't put yourself at disadvantage with a locked-down six-year out of date card. Tenstorrent Blackhole exists now, btw.

Be serious now. Plenty of useful, privacy-required queries can be run with 24GB of VRAM, especially given the existence of e.g. Gemma 3 27B and the heavy NVIDIA-targeted optimisation work that has occurred.

The Tenstorrent cards exist, but are low in availability and the software is comparatively nonexistent. I'm excited for them too, but at the end of the day, I can buy a used 3090 today and do useful work with it, while the same is not true of TT yet.

Re: The Llama 4 herd

#585

Post by op00to »

parhamn wrote: post #112
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

"I'd normally lie to you but," is not what's actually implied when "Honestly," is used conversationally. If you overthink things like this you're going to have a tough time communicating with people.

I'm not saying you need to stop using it, but I prefer to not indicate that in some situations I would lie, but in this one specifically I won't. I communicate with customers constantly in my job, and my integrity and reputation is most important to me. If I'm going to lie, I'd rather not call attention to it.

When an LLM says "honestly", it's just stupid. An LLM can't "lie".

Re: The Llama 4 herd

#586

Post by op00to »

lucianbr wrote: post #115
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

"Honestly" and "literally" are now used in English for emphasis. I dislike this, but it's the current reality. I don't think there's any way to get back to only using them with their original meanings.

I don't think anyone needs to change their language. I understand that it's a common way to indicate candor, but it's hilariously inappropriate for a computer to say "some times I might lie to you to save your feelings, but this time, you really are ugly and you need to know."

Re: The Llama 4 herd

#587

Post by op00to »

andrewstuart wrote: post #78
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

Or when it asks you questions. The only time an LLM should ask questions is to clarify information. A word processor doesn’t want to chit chat about what I’m writing about, nor should an LLM. Unless it is specifically playing an interactive role of some sort like a virtual friend.

ChatGPT is very casual with asking questions, and FRANKLY, I enjoy getting into a little bit of a daydream with it from time to time. It's taken the place of falling into a Wikipedia hole. Not sure if that's something that's good or bad.

Re: The Llama 4 herd

#588

Post by [deleted] »

Auracle wrote: post #500
paxys wrote: post #230

Earlier quoted context omitted.

That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

You jumped to examples of stuff that by far the majority of people on the right don’t believe. If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?” The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make…

[deleted]

Re: The Llama 4 herd

#589

Post by dpkirchner »

Auracle wrote: post #500
paxys wrote: post #230

Earlier quoted context omitted.

That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

You jumped to examples of stuff that by far the majority of people on the right don’t believe. If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?” The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make…

> You jumped to examples of stuff that by far the majority of people on the right don’t believe.

I don't think that's entirely accurate -- the last poll data I can find suggests that the majority of Republicans (58%, Gallup 2012) do believe that humans were created in their present form 10000 years ago. Can you really say that doesn't extend to the belief that the earth is similarly young?

Re: The Llama 4 herd

#590

Post by mcculley »

parhamn wrote: post #112
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

"I'd normally lie to you but," is not what's actually implied when "Honestly," is used conversationally. If you overthink things like this you're going to have a tough time communicating with people.

“Candidly” is more precise where people usually use “honestly”.
Post reply on HN