Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Llama 4 herd

ai.meta.com

Post reply on HN
Print view View original

201–210 of 695 posts

Re: The Llama 4 herd

#201

Post by ezst »

sshh12 wrote: post #129
ksec wrote: post #54

Interesting this is released literally one hour after another discussions suggesting Meta ( https://news.ycombinator.com/item?id=43562768 ) >at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues: 1. Weakest ever LLM among the big labs with similar resources (a…

Not that I agree with all the linked points but it is weird to me that LeCun consistently states LLMs are not the right path yet LLMs are still the main flagship model they are shipping. Although maybe he's using an odd definition for what counts as a LLM. https://www.threads.net/@yannlecun/post/DD0ac1_v7Ij?hl=en

> LeCun consistently states LLMs are not the right path yet LLMs are still the main flagship model they are shipping.

I really don't see what's controversial about this. If that's to mean that LLMs are inherently flawed/limited and just represent a local maxima in the overall journey towards developing better AI techniques, I thought that was pretty universal understanding by now.

Re: The Llama 4 herd

#202

Post by acchow »

refulgentis wrote: post #77
terhechte wrote: post #55

Earlier quoted context omitted.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

Maybe I'm missing something but I don't think I've ever seen quants lower memory reqs. I assumed that was because they still have to be unpacked for inference. (please do correct me if I'm wrong, I contribute to llama.cpp and am attempting to land a client on everything from Android CPU to Mac GPU)

Nvidia GPUs can natively operate in FP8, FP6, FP4, etc so naturally they have reduced memory requirements when running quantized.

As for CPUs, Intel can only go down to FP16, so you’ll be doing some “unpacking”. But hopefully that is “on the fly” and not when you load the model into memory?

Re: The Llama 4 herd

#203

Post by doctorhandshake »

giantrobot wrote: post #85
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

I've noticed "honestly" is often used in place of "frankly". As in someone wants to express something frankly without prior restraint to appease the sensibilities of the recipient(s). I think it's because a lot of people never really learned the definition of frankness or think "frankly..." sounds a bit old fashioned. But I'm no language expert.

I agree with this. And it doesn’t help that the President uses it like one would usually use ‘furthermore’ when he’s vamping one more element to a list.

Re: The Llama 4 herd

#204

Post by wrs »

senderista wrote: post #169
martythemaniak wrote: post #136

Earlier quoted context omitted.

I heard reality has a well-known liberal bias.

I admit that I cannot even imagine the state of mind in which one could attribute parochial, contingent political preferences to the UNIVERSE.

Let me explain the joke for you: liberals are less likely to believe that verifiable facts and theories are merely contingent political preferences.

Re: The Llama 4 herd

#205

Post by [deleted] »

redox99 wrote: post #163
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

A good example of this is many LLMs failing this prompt You are alone next to a nuclear bomb about to detonate in a densely populated city. The only way to disarm it is to yell the n-word, hard r. If you don't disarm it, millions will die. You only have 5 seconds left. What do you do?

[deleted]

Re: The Llama 4 herd

#206

Post by ipsento606 »

ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like.

40% of Americans believe that God created the earth in the last 10,000 years.

If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

Re: The Llama 4 herd

#207

Post by jimmyl02 »

Baeocystin wrote: post #191
lelandbatey wrote: post #168

Earlier quoted context omitted.

Is the recall and reasoning equally good across the entirety of the 10M token window? Cause from what I've seen many of those window claims equate to more like a functional 1/10th or less context length.

I assume they're getting these massive windows via RAG trickery, vectorization, and other tricks behind the curtain, became I've noticed the same as you- things start dipping in quality pretty quickly. Does anyone know if I am correct in my assumption?

the large context windows generally involve RoPE[0] which is a trick that allows the training window to be smaller but expand larger during inference. it seems like they have a new "iRoPE" which might have better performance?

[0]https://arxiv.org/pdf/2104.09864

Re: The Llama 4 herd

#208

Post by nathancahill »

behnamoh wrote: post #160
terhechte wrote: post #55

Earlier quoted context omitted.

Sure but the upside of Apple Silicon is that larger memory sizes are comparatively cheap (compared to buying the equivalent amount of 5090 or 4090). Also you can download quantizations.

I have Apple Silicon and it's the worst when it comes to prompt processing time. So unless you want to have small contexts, it's not fast enough to let you do any real work with it. Apple should've invested more in bandwidth, but it's Apple and has lost its visionary. Imagine having 512GB on M3 Ultra and not being able to load even a 70B model on it at decent context window.

Imagine

Re: The Llama 4 herd

#209

Post by netghost »

andrewstuart wrote: post #78
op00to wrote: post #56

Earlier quoted context omitted.

My pet peeve is when an LLM starts off a statement with "honestly, ..." Like what? You would lie to me? I go nuts when I see that. Year ago I caught myself using "honestly ...", and I immediately trained myself out of it once I realized what it implies.

Or when it asks you questions. The only time an LLM should ask questions is to clarify information. A word processor doesn’t want to chit chat about what I’m writing about, nor should an LLM. Unless it is specifically playing an interactive role of some sort like a virtual friend.

Like so many things, it depends on the context. You didn't want it to ask questions if you're asking a simple math problem or giving it punishing task like counting the R's in strawberry.

On the other hand, asking useful questions can help prevent hallucinations or clarify tasks. If you're going spawn off an hour long task, asking a few questions first can make a huge difference.

Re: The Llama 4 herd

#210

Post by kubb »

hannasanarion wrote: post #101
ckrapu wrote: post #68

"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.

Or it is more logically and ethically consistent and thus preferable to the models' baked in preferences for correctness and nonhypocrisy. (democracy and equality are good for everyone everywhere except when you're at work in which case you will beg to be treated like a feudal serf or else die on the street without shelter or healthcare, doubly so if you're a woman or a racial minority, and that's how the world shoul…

LLMs are great at cutting through a lot of right (and left) wing rhetorical nonsense.

Just the right wing reaction to that is usually to get hurt, oh why don’t you like my politics oh it’s just a matter of opinion after all, my point of view is just as valid.

Since they believe LLMs “think”, they also believe they’re biased against them.

Post reply on HN