Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

91–100 of 695 posts

Re: The Llama 4 herd

#91

Post by terhechte »

refulgentis wrote: post #77
terhechte wrote: post #55

Earlier quoted context omitted.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

Maybe I'm missing something but I don't think I've ever seen quants lower memory reqs. I assumed that was because they still have to be unpacked for inference. (please do correct me if I'm wrong, I contribute to llama.cpp and am attempting to land a client on everything from Android CPU to Mac GPU)

I just loaded two models of different quants into LM Studio:

qwen 2.5 coder 1.5b @ q4_k_m: 1.21 GB memory

qwen 2.5 coder 1.5b @ q8: 1.83 GB memory

I always assumed this to be the case (also because of the smaller download sizes) but never really thought about it.

Re: The Llama 4 herd

#92

Post by lostmsu »

qwertox wrote: post #63
laborcontract wrote: post #29

General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…

Llama 4 Scout, Maximum context length: 10M tokens. This is a nice development.

How did they achieve such a long window and what are the memory requirements to utilize it?

Re: The Llama 4 herd

#93

Post by falcor84 »

ksec wrote: post #54

Interesting this is released literally one hour after another discussions suggesting Meta ( https://news.ycombinator.com/item?id=43562768 ) >at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues: 1. Weakest ever LLM among the big labs with similar resources (a…

I don't understand what LeCun is trying to say. Why does he give an interview saying that LLM's are almost obsolete just when they're about to release a model that increases the SotA context length by an order of magnitude? It's almost like a Dr. Jekyll and Mr. Hyde situation.

Re: The Llama 4 herd

#94

Post by mdp2021 »

andrewstuart wrote: post #20

Self hosting LLMs will explode in popularity over next 12 months. Open models are made much more interesting and exciting and relevant by new generations of AI focused hardware such as the AMD Strix Halo and Apple Mac Studio M3. GPUs have failed to meet the demands for lower cost and more memory so APUs look like the future for self hosted LLMs.

> new generations of AI focused hardware

Some benchmarks are not encouraging. See e.g. https://www.hardware-corner.net/mac-studio-m3-ultra-deepseek...

That «AI focused hardware» will either have extremely fast memory, and cost prohibitively, or have reasonable costs, and limits that are to be assessed.

Re: The Llama 4 herd

#95

Post by throwanem »

rpastuszak wrote: post #71
mdp2021 wrote: post #33

Earlier quoted context omitted.

Well, it is a speaker (writer) after all. It has to use some way to refer to itself.

I don't think that's true. It's more of a function on how these models are trained (remember the older pre-ChatGPT clients?) Most of the software I use doesn't need to refer it itself in the first person. Pretending what we're speaking with an agent is more of a UX/marketing decision rather than a technical/logical constraint.

I'm not sure about that. What happens if you "turn down the weight" (cf. https://www.anthropic.com/news/golden-gate-claude) for self-concept, expressed in the use not of first-person pronouns but "the first person" as a thing that exists? Do "I" and "me" get replaced with "this one" like someone doing depersonalization kink, or does it become like Wittgenstein's lion in that we can no longer confidently parse even its valid utterances? Does it lose coherence entirely, or does something stranger happen?

It isn't an experiment I have the resources or the knowledge to run, but I hope someone does and reports the results.

Re: The Llama 4 herd

#97

Post by Melklington »

ksec wrote: post #54

Interesting this is released literally one hour after another discussions suggesting Meta ( https://news.ycombinator.com/item?id=43562768 ) >at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues: 1. Weakest ever LLM among the big labs with similar resources (a…

LeCun doesn't believe in LLM Architecture anyway.

Could easily be that he just researches bleeding edge with his team and others work on Llama + doing experiements with new technices on it.

Any blog post or yt docu going into detail how they work?

Re: The Llama 4 herd

#98

Post by UltraSane »

qntmfred wrote: post #73
mrbonner wrote: post #43

What an electrifying time to be alive! The last era that felt even remotely this dynamic was during the explosive rise of JavaScript frameworks—when it seemed like a new one dropped every quarter. Back then, though, the vibe was more like, “Ugh, another framework to learn?” Fast forward to now, and innovation is sprinting forward again—but this time, it feels like a thrilling ride we can’t wait to be part of.

I know what you mean in terms of frantic pace of "new stuff" coming out, but I winced at the comparison of innovation in AI to mere web development tooling.

Yes. LLMs and latent spaces are vastly more interesting.

Re: The Llama 4 herd

#99

Post by sejje »

ANewFormation wrote: post #64
mdp2021 wrote: post #33

Earlier quoted context omitted.

Well, it is a speaker (writer) after all. It has to use some way to refer to itself.

So is a command prompt.

Command prompts don't speak English.

Command prompts don't get asked questions like "What do you think about [topic]?" and have to generate a response based on their study of human-written texts.

Post reply on HN