Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

421–430 of 695 posts

Re: The Llama 4 herd

#421

Post by RandyOrion »

I guess I have to say thank you Meta?

A somewhat sad rant below.

Deepseek starts a toxic trend of providing super, super large MoE. And MoE is famous for being parameter-inefficient, which is unfriendly to normal consumer hardware with limited vram.

The super large size of LLM also disables nearly every people from doing meaningful development on these models. R1-1776 is the only fine-tune variation of R1 that makes some noise, and it's by a corp not some random individual.

In this release, the smallest Llama 4 model is over 100B, which is not small by any means, and will prevent people from fine-tuning as well.

On top of that, to access llama models on hugging face has become notoriously hard because of 'permission' issues. See details in https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/dis...

Yeah, I personally don't really see the point of releasing large MoEs. I'll stick to small and dense LLMs from Qwen, Mistral, Microsoft, Google and others.

Edit: This comment got downvoted, too. Please explain your reason before doing that.

Re: The Llama 4 herd

#422

Post by ckrapu »

nwienert wrote: post #255
LeafItAlone wrote: post #214

Earlier quoted context omitted.

>Worldwide centrist and conservative groups account for 60%+ of the population. Source? >See also for example recent USAID gutting and reasons behind it. A very politically motivated act does not prove anything about the “traditional structure of Internet media which reflects the underlying population very poorly”.

China, Africa, India, Vietnam, Philippines, Russia? Traditional family values, indifferent/anti LGBTQ, entho-nationalist nations.

You're conflating culture war issues with ideology.

For most of the world, left and right are economic axes despite the American corporate media's attempts to convince you that the 0.1% of crossdressers are more important than making sure you and your family get a fair wage and clean air.

Re: The Llama 4 herd

#423

Post by EGreg »

Can we somehow load these inside node.js?

What is the easiest way to load them remotely? Huggingface Spaces? Google AI Studio?

I am teaching a course on AI to non-technical students, and I wanted the students to have a minimal setup: which in this case would be:

1) Browser with JS (simple folder of HTML, CSS) and Tensorflow.js that can run models like Blazeface for face recognition, eye tracking etc. (available since 2019)

2) Node.js with everything baked in (javascript) and use a CDN like CloudFront with tunnel to serve it to the web

3) So if they download models to their computer, how would they run them? Is it possible to run the smallest LLaMa locally? Or any GGUF models in JS? Or they have to have Python and PyTorch?

PS: Here is what the class looks like: https://vimeo.com/1060576298/c5693047e0?share=copy

Re: The Llama 4 herd

#424

Post by anon373839 »

terhechte wrote: post #26

The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked a local 7B model (qwen 2.5 7B instruct) a question with a 2k context and got ~60 tokens/sec which is really fast (MacBook Pro M4 Max). So this could hit 30 token/sec. Time to first…

Unless I'm missing something, I don't really think it looks that attractive. They're comparing it to Mistral Small 24B and Gemma 3 27B and post numbers showing that is a little better than those models. But at 4x the memory footprint, is it worth it? (Personally, I was hoping to see Meta's version of a 24-32B dense model since that size is clearly very capable, or something like an updated version of Mixtral 8x7B.)

Re: The Llama 4 herd

#425

Post by tintor »

terhechte wrote: post #26

The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked a local 7B model (qwen 2.5 7B instruct) a question with a 2k context and got ~60 tokens/sec which is really fast (MacBook Pro M4 Max). So this could hit 30 token/sec. Time to first…

Not as fast as other 17B models if it has to attend to 10M context window.

Re: The Llama 4 herd

#426

Post by fumeux_fume »

ipsento606 wrote: post #206
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

Bias doesn't matter as long as you clearly state your priors.

Re: The Llama 4 herd

#427

Post by b0ner_t0ner »

CSMastermind wrote: post #362
mrbonner wrote: post #43

What an electrifying time to be alive! The last era that felt even remotely this dynamic was during the explosive rise of JavaScript frameworks—when it seemed like a new one dropped every quarter. Back then, though, the vibe was more like, “Ugh, another framework to learn?” Fast forward to now, and innovation is sprinting forward again—but this time, it feels like a thrilling ride we can’t wait to be part of.

I lived through the explosion of JavaScript frameworks and this feels way bigger to me. For me at least it feels closer to the rise of the early internet. Reminds me of 1996.

It'll be worse actually, with all the vibe coders out there: https://www.reddit.com/r/vibecoding/

Re: The Llama 4 herd

#428

Post by wat10000 »

dcsommer wrote: post #302
ipsento606 wrote: post #206

Earlier quoted context omitted.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

> 40% of Americans believe that God created the earth in the last 10,000 years. Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution. https://www.pewresearch.org/religion/2019/02/06/the-evolutio...

The fact that YEC is incompatible with human evolution doesn’t mean people can’t believe both. Especially since “god guided human evolution” can mean something very different than actual evolution.

Re: The Llama 4 herd

#430

Post by tqi »

> Our testing shows that Llama 4 responds with strong political lean at a rate comparable to Grok (and at half of the rate of Llama 3.3) on a contentious set of political or social topics. While we are making progress, we know we have more work to do and will continue to drive this rate further down.

My experience is that these subjective benchmarks are completely meaningless, because the researchers involved have a strong incentive (promotions, discretionary equity) to cherrypick measures that they can easily improve.

Post reply on HN