The Llama 4 herd • Hacker News Board

Re: The Llama 4 herd

#71

Post by rpastuszak » Sat, Apr 05, 2025, 7:01 PM UTC

mdp2021 wrote: post #33

andrewstuart wrote: post #25

Earlier quoted context omitted.

Personally I’d prefer that LLMs did not refer to themselves as “I”. It’s software, not an “I”.

Well, it is a speaker (writer) after all. It has to use some way to refer to itself.

I don't think that's true. It's more of a function on how these models are trained (remember the older pre-ChatGPT clients?)

Most of the software I use doesn't need to refer it itself in the first person. Pretending what we're speaking with an agent is more of a UX/marketing decision rather than a technical/logical constraint.

Re: The Llama 4 herd

#72

Post by philipwhiuk » Sat, Apr 05, 2025, 7:01 PM UTC

artninja1988 wrote: post #65

Thank you meta for open sourcing! Will there be a llama with native image output similar to 4os? Would be huge

Probably to head off allegations of profiting from breach of copyright.

Re: The Llama 4 herd

#73

Post by qntmfred » Sat, Apr 05, 2025, 7:01 PM UTC

mrbonner wrote: post #43

What an electrifying time to be alive! The last era that felt even remotely this dynamic was during the explosive rise of JavaScript frameworks—when it seemed like a new one dropped every quarter. Back then, though, the vibe was more like, “Ugh, another framework to learn?” Fast forward to now, and innovation is sprinting forward again—but this time, it feels like a thrilling ride we can’t wait to be part of.

I know what you mean in terms of frantic pace of "new stuff" coming out, but I winced at the comparison of innovation in AI to mere web development tooling.

Re: The Llama 4 herd

#74

Post by nattaylor » Sat, Apr 05, 2025, 7:02 PM UTC

Is pre-training in FP8 new?

Also, 10M input token context is insane!

EDIT: https://huggingface.co/meta-llama/Llama-3.1-405B is BF16 so yes, it seems training in FP8 is new.

Re: The Llama 4 herd

#75

Post by mdp2021 » Sat, Apr 05, 2025, 7:02 PM UTC

ANewFormation wrote: post #64

mdp2021 wrote: post #33

Earlier quoted context omitted.

Well, it is a speaker (writer) after all. It has to use some way to refer to itself.

So is a command prompt.

Agnew, if you converse with your command prompt we are glad you came here for a break ;)

Re: The Llama 4 herd

#76

Post by terhechte » Sat, Apr 05, 2025, 7:02 PM UTC

terhechte wrote: post #26

The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked a local 7B model (qwen 2.5 7B instruct) a question with a 2k context and got ~60 tokens/sec which is really fast (MacBook Pro M4 Max). So this could hit 30 token/sec. Time to first…

To add, they say about the 400B "Maverick" model:

> while achieving comparable results to the new DeepSeek v3 on reasoning and coding

If that's true, it will certainly be interesting for some to load up this model on a private M3 Studio 512GB. Response time will be fast enough for interaction in Roo Code or Cline. Prompt processing is a bit slower but could be manageable depending on how much code context is given to the model.

The upside being that it can be used on codebases without having to share any code with a LLM provider.

Re: The Llama 4 herd

#77

Post by refulgentis » Sat, Apr 05, 2025, 7:02 PM UTC

terhechte wrote: post #55

scosman wrote: post #37

Earlier quoted context omitted.

At 109b params you’ll need a ton of memory. We’ll have to wait for evals of the quants to know how much.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

Maybe I'm missing something but I don't think I've ever seen quants lower memory reqs. I assumed that was because they still have to be unpacked for inference. (please do correct me if I'm wrong, I contribute to llama.cpp and am attempting to land a client on everything from Android CPU to Mac GPU)

Re: The Llama 4 herd

#78

Post by andrewstuart » Sat, Apr 05, 2025, 7:03 PM UTC

op00to wrote: post #56

andrewstuart wrote: post #25

Earlier quoted context omitted.

Personally I’d prefer that LLMs did not refer to themselves as “I”. It’s software, not an “I”.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

Or when it asks you questions.

The only time an LLM should ask questions is to clarify information. A word processor doesn’t want to chit chat about what I’m writing about, nor should an LLM.

Unless it is specifically playing an interactive role of some sort like a virtual friend.

Re: The Llama 4 herd

#79

Post by NoahKAndrews » Sat, Apr 05, 2025, 7:03 PM UTC

spwa4 wrote: post #35

I hope this time multimodal includes multimodal outputs!

Nope

Re: The Llama 4 herd

#80

Post by mrbonner » Sat, Apr 05, 2025, 7:04 PM UTC

misnome wrote: post #60

mrbonner wrote: post #43

What an electrifying time to be alive! The last era that felt even remotely this dynamic was during the explosive rise of JavaScript frameworks—when it seemed like a new one dropped every quarter. Back then, though, the vibe was more like, “Ugh, another framework to learn?” Fast forward to now, and innovation is sprinting forward again—but this time, it feels like a thrilling ride we can’t wait to be part of.

Did “A new javascript framework de jour every quarter” ever stop happening?

No, but apparently people stop caring and chasing the wagon.