Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

221–230 of 695 posts

Re: The Llama 4 herd

#221

Post by spoll »

g-mork wrote: post #182
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

Worldwide centrist and conservative groups account for 60%+ of the population. The training data bias is due to the traditional structure of Internet media which reflects the underlying population very poorly. See also for example recent USAID gutting and reasons behind it.

Presumably you could also argue that 60 plus percent is made up by centrist and leftist groups, centrism being what it is.

Re: The Llama 4 herd

#222

Post by randunel »

mdp2021 wrote: post #197
nrp wrote: post #127

Earlier quoted context omitted.

We took the Ryzen AI Max, which is nominally a high-end laptop processor, and built it into a standard PC form factor (Mini-ITX). It’s a more open/extensible mini PC using mobile technology.

And given that some people are afraid of malicious software in some brands of mini-PCs on the market, to have some more trusted product around will also be an asset.

Lenovo backdoors as preinstalled software, including their own TLS certificate authorities.

Name whom you're referring to every time!

Re: The Llama 4 herd

#223

Post by mirekrusin »

behnamoh wrote: post #160
terhechte wrote: post #55

Earlier quoted context omitted.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

I have Apple Silicon and it's the worst when it comes to prompt processing time. So unless you want to have small contexts, it's not fast enough to let you do any real work with it. Apple should've invested more in bandwidth, but it's Apple and has lost its visionary. Imagine having 512GB on M3 Ultra and not being able to load even a 70B model on it at decent context window.

At 17B active params MoE should be much faster than monolithic 70B, right?

Re: The Llama 4 herd

#224

Post by voidspark »

accrual wrote: post #184
rfoo wrote: post #106

From model cards, suggested system prompt: > You are Llama 4. Your knowledge cutoff date is August 2024. You speak Arabic, English, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, and Vietnamese. Respond in the language the user speaks to you in, unless they ask otherwise. It's interesting that there's no single one of CJK languages mentioned. I'm tempted to call this a racist model ev…

Isn't there a vast quantity of relevant information in CJK languages? I remember reading some models even "think" in other languages where there might be more detail before outputting in the target language.

The model wasn't trained on those languages (yet). The only possible explanation is racism. The model is also racist against Russians and Icelanders.

Re: The Llama 4 herd

#225

Post by 1ucky »

behnamoh wrote: post #160
terhechte wrote: post #55

Earlier quoted context omitted.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

I have Apple Silicon and it's the worst when it comes to prompt processing time. So unless you want to have small contexts, it's not fast enough to let you do any real work with it. Apple should've invested more in bandwidth, but it's Apple and has lost its visionary. Imagine having 512GB on M3 Ultra and not being able to load even a 70B model on it at decent context window.

Prompt preprocessing is heavily compute-bound, so relying significantly on processing capabilities. Bandwidth mostly affects token generation speed.

Re: The Llama 4 herd

#226

Post by senderista »

wrs wrote: post #204
senderista wrote: post #169

Earlier quoted context omitted.

I admit that I cannot even imagine the state of mind in which one could attribute parochial, contingent political preferences to the UNIVERSE.

Let me explain the joke for you: liberals are less likely to believe that verifiable facts and theories are merely contingent political preferences.

I see leftists denying inconvenient facts just as much as rightists. It's just the inevitable product of a tribal mentality, the tribe doesn't matter.

Re: The Llama 4 herd

#227

Post by kgwgk »

terhechte wrote: post #109
anoncareer0212 wrote: post #89

Earlier quoted context omitted.

Small point of order: bit slower might not set expectations accurately. You noted in a previous post in the same thread[^1] that we'd expect about a 1 minute per 10K tokens(!) prompt processing time with the smaller model. I agree, and contribute to llama.cpp. If anything, that is quite generous. [^1] https://news.ycombinator.com/item?id=43595888

I don't think the time grows linearly. The more context the slower (at least in my experience because the system has to throttle). I just tried 2k tokens in the same model that I used for the 120k test some weeks ago and processing took 12 sec to first token (qwen 2.5 32b q8).

> The more context the slower

It seems the other way around?

120k : 2k = 600s : 10s

Re: The Llama 4 herd

#228

Post by Centigonal »

comex wrote: post #49

So how does the 10M token context size actually work? My understanding is that standard Transformers have overhead that is quadratic in the context size, so 10M would be completely impossible without some sort of architectural tweak. This is not the first model to have a huge context size, e.g. Gemini has 2M, but my understanding is that the previous ones have generally been proprietary, without public weights or arc…

Gemini likely uses something based on RingAttention to achieve its long context sizes. This requires massive inference clusters, and can't be the same approach llama4 is using. Very curious how llama4 achieves its context length.

Re: The Llama 4 herd

#229

Post by throwanem »

ilove_banh_mi wrote: post #198
throwanem wrote: post #69

Earlier quoted context omitted.

Castrated, if you're trying way too hard (and not well) to avoid getting called on that overly emotive metaphor: a capon is a gelded rooster.

There is a key distinction and context: caponation has a productive purpose from the pov of farmers and their desired profits.

I gather the term of art is "caponization," but that's a cavil. For something that is not born with testes or indeed at all, to describe it with this metaphor is very silly and does nothing to elucidate whatever it is you're actually getting at.

Re: The Llama 4 herd

#230

Post by paxys »

vessenes wrote: post #220
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

Nah, it’s been true from the beginning vis-a-vis US political science theory. That is, if you deliver something like https://www.pewresearch.org/politics/quiz/political-typology... To models from GPT-3 on you get highly “liberal” per Pew’s designations. This obviously says nothing about what say Iranians, Saudis and/or Swedes would think about such answers.

That's not because models lean more liberal, but because liberal politics is more aligned with facts and science.

Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

Post reply on HN