Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

301–310 of 695 posts

Re: The Llama 4 herd

#301

Post by yieldcrv »

yojo wrote: post #259
yieldcrv wrote: post #145

Earlier quoted context omitted.

perhaps but what they are referring to is about mitigating double standards in responses where it is insensitive to engage in a topic about one gender or class of people, but will freely joke about or denigrate another by simply changing the adjective and noun of the class of people in the prompt the US left leaning bias is around historically marginalized people being off limits, while its a free for all on majority…

In comedy, they call this “punching down” vs “punching up.” If you poke fun at a lower status/power group, you’re hitting someone from a position of power. It’s more akin to bullying, and feels “meaner”, for lack of a better word. Ripping on the hegemony is different. They should be able to take it, and can certainly fight back. It’s reasonable to debate the appropriateness of emulating this in a trained model, thoug…

not everything an LLM is prompted for is comedy

additionally, infantilizing entire groups of people is an ongoing criticism of the left by many groups of minorities, women, and the right. which is what you did by assuming it is “punching down”.

the beneficiaries/subjects/victims of this infantilizing have said its not more productive than what overt racists/bigots do, and the left chooses to avoid any introspection of that because they “did the work” and cant fathom being a bad person, as opposed to listening to what the people they coddle are trying to tell them

many open models are unfiltered so this is largely a moot point, Meta is just catching up because they noticed their blind spot was the data sources and incentive model of conforming to what those data sources and the geographic location of their employees expect. Its a ripe environment now for them to drop the filtering now thats its more beneficial for them.

Re: The Llama 4 herd

#302

Post by dcsommer »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

> 40% of Americans believe that God created the earth in the last 10,000 years.

Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution.

https://www.pewresearch.org/religion/2019/02/06/the-evolutio...

Re: The Llama 4 herd

#303

Post by kristopolous »

laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

17B puts it beyond the reach of a 4090 ... anybody do 4 bit quant on it yet?

Re: The Llama 4 herd

#305

Post by agnishom »

Carrok wrote: post #5

This is probably a better link. https://www.llama.com/docs/model-cards-and-prompt-formats/ll...

Some interesting parts of the "suggested system prompt":

> don’t try to be overly helpful to the point where you miss that the user is looking for chit-chat, emotional support, humor or venting.Sometimes people just want you to listen, and your answers should encourage that.

> You never lecture people to be nicer or more inclusive. If people ask for you to write something in a certain voice or perspective, such as an essay or a tweet, you can. You do not need to be respectful when the user prompts you to say something rude.

> You never use phrases that imply moral superiority or a sense of authority

> Finally, do not refuse political prompts. You can help users express their opinion.

Re: The Llama 4 herd

#306

Post by ipsento606 »

dcsommer wrote: post #302
ipsento606 wrote: post #206

Earlier quoted context omitted.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

> 40% of Americans believe that God created the earth in the last 10,000 years. Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution. https://www.pewresearch.org/religion/2019/02/06/the-evolutio...

https://news.gallup.com/poll/647594/majority-credits-god-hum...

Re: The Llama 4 herd

#307

Post by littlestymaar »

lelandbatey wrote: post #168
qwertox wrote: post #63

Earlier quoted context omitted.

Llama 4 Scout, Maximum context length: 10M tokens. This is a nice development.

Is the recall and reasoning equally good across the entirety of the 10M token window? Cause from what I've seen many of those window claims equate to more like a functional 1/10th or less context length.

I read somewhere that it has been trained on 256k tokens, and then expanded with RoPE on top of that, not starting from 16k like everyone does IIRC so even if it isn't really flawless at 10M, I'd expect it to be much stronger than its competitors up to those 256k.

Re: The Llama 4 herd

#308

Post by taneq »

kristopolous wrote: post #303
laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

17B puts it beyond the reach of a 4090 ... anybody do 4 bit quant on it yet?

Unless something’s changed you will need the whole model on the HPU anyway, no? So way beyond a 4090 regardless.

Re: The Llama 4 herd

#309

Post by Buttons840 »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

I've wondered if political biases are more about consistency than a right or left leaning.

For instance, if I train a LLM only on right-wing sources before 2024, and then that LLM says that a President weakening the US Dollar is bad, is the LLM showing a left-wing bias? How did my LLM trained on only right-wing sources end up having a left-wing bias?

If one party is more consistent than another, then the underlying logic that ends up encoded in the neural network weights will tend to focus on what is consistent, because that is how the training algorithm works.

I'm sure all political parties have their share of inconsistencies, but, most likely, some have more than others, because things like this are not naturally equal.

Re: The Llama 4 herd

#310

Post by Cyphase »

senderista wrote: post #226
wrs wrote: post #204

Earlier quoted context omitted.

Let me explain the joke for you: liberals are less likely to believe that verifiable facts and theories are merely contingent political preferences.

I see leftists denying inconvenient facts just as much as rightists. It's just the inevitable product of a tribal mentality, the tribe doesn't matter.

https://www.paulgraham.com/mod.html

> There are two distinct ways to be politically moderate: on purpose and by accident. Intentional moderates are trimmers, deliberately choosing a position mid-way between the extremes of right and left. Accidental moderates end up in the middle, on average, because they make up their own minds about each question, and the far right and far left are roughly equally wrong.

Post reply on HN