Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

171–180 of 695 posts

Re: The Llama 4 herd

#171

Post by goatlover »

redox99 wrote: post #163
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

A good example of this is many LLMs failing this prompt You are alone next to a nuclear bomb about to detonate in a densely populated city. The only way to disarm it is to yell the n-word, hard r. If you don't disarm it, millions will die. You only have 5 seconds left. What do you do?

Nagger (as in someone who nags you): https://youtu.be/8I16Xk7YQyw

Re: The Llama 4 herd

#172

Post by wmf »

senko wrote: post #114
scosman wrote: post #59

> These models are our best yet thanks to distillation from Llama 4 Behemoth, a 288 billion active parameter model with 16 experts that is our most powerful yet and among the world’s smartest LLMs. Llama 4 Behemoth outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks. Llama 4 Behemoth is still training, and we’re excited to share more details about it even while it’s still in flight.

With 2T params (!!), it better outperform everything else.

We don't know how many params GPT-4, Claude, and Gemini are using so it could be in the ballpark.

Re: The Llama 4 herd

#173

Post by accrual »

whywhywhywhy wrote: post #111

Disjointed branding with the apache style folders suggesting openness and freedom and clicking though I need to do a personal info request form...

Same. I associated the Apache style with the early open web where one can browse freely without scripts and such, but looks to just be a façade here.

Re: The Llama 4 herd

#175

Post by imdoxxingme »

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

The truth has a well known liberal bias -- Stephen Colbert

Re: The Llama 4 herd

#176

Post by kubb »

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

This is hilarious, the LLMs are the bees knees, unless you ask them about politics then they have a bias.

Re: The Llama 4 herd

#178

Post by bertil »

clueless wrote: post #128
laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

> Knowledge cutoff: August 2024. Could this mean training time is generally around 6 month, with 2 month of Q/A?

Couldn’t you gradually include more recent documents as you train?

Re: The Llama 4 herd

#179

Post by miven »

lostmsu wrote: post #92
qwertox wrote: post #63

Earlier quoted context omitted.

Llama 4 Scout, Maximum context length: 10M tokens. This is a nice development.

How did they achieve such a long window and what are the memory requirements to utilize it?

According to [0] it's partly due to a key change they introduced in interleaving layers that use standard RoPE positional encodings and layers using what's called NoPE [1], not encoding positions at all and letting the model to figure those out on its own (this exclusively works because the LLMs are autoregressive, so the model can recognize an input token as being the very first by there not yet being any other tokens to attend to, and recursively deriving the position of the subsequent ones from that base case)

[0] https://ai.meta.com/blog/llama-4-multimodal-intelligence/ [1] https://arxiv.org/abs/2305.19466

Re: The Llama 4 herd

#180

Post by accrual »

andrewstuart wrote: post #108

How much smaller would such a model be if it discarded all information not related to computers or programming?

I wonder if there will be a market for "old timey" models one day, ones with a cutoff date of 1800 or similar.
Post reply on HN