Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens if we remove 50 percent of Llama?

neuralmagic.com

Post reply on HN
Print view View original

111–120 of 139 posts

Re: What happens if we remove 50 percent of Llama?

#111

Post by Loughla »

LoganDark wrote: post #73
v3ss0n wrote: post #60

Earlier quoted context omitted.

I didn't say low intellect but , as also a functioning autist (as most of us are) i know myself that i am something wrong compare to other people who are quite different.

I don't think you are something wrong, I think it's wonderful that brains can be so different. I'm fascinated by every type of neurodivergence. You should be proud of what you are, not ashamed of being "something wrong".

That statement comes from a place of amazing privilege. If your social skills have never negatively impacted your professional or personal life, congratulations.

For the rest of us, it's not always a gift. It can be (for me that's analytical thinking and technical writing). But it can also be an absolute curse.

Re: What happens if we remove 50 percent of Llama?

#112

Post by menaerus »

robotresearcher wrote: post #110
menaerus wrote: post #54

Earlier quoted context omitted.

Yeah, I also think that the ~5k price is quite hefty. It's difficult for me to imagine that running sizeable LLMs on commodity/consumer hardware will be possible without another breakthrough in the field. The prices of GPUs I wouldn't expect to fall if technology proves its worthiness.

You're predicting the price of computer chips will not fall? They're just about the most price-fally truly useful thing in history.

Yes, I am.

Re: What happens if we remove 50 percent of Llama?

#113

Post by ben_w »

robotresearcher wrote: post #110
menaerus wrote: post #54

Earlier quoted context omitted.

Yeah, I also think that the ~5k price is quite hefty. It's difficult for me to imagine that running sizeable LLMs on commodity/consumer hardware will be possible without another breakthrough in the field. The prices of GPUs I wouldn't expect to fall if technology proves its worthiness.

You're predicting the price of computer chips will not fall? They're just about the most price-fally truly useful thing in history.

They have been to date.

Massive increases in demand due to this stuff being really really useful can cause prices to go up even for existing chips (NVIDIA is basically printing money as they can sell all they can make at for as much money as the buyers can get from the investors). I have vague memories of something like this happening with RAM in the late 90s, but perhaps it was just Mac RAM because the Apple market was always its own weird oddity (the Performa 5200 I bought around then was also available in the second hand listings on one of the magazines for twice what I paid for it).

Likewise prices can go up from global trade wars, e.g. like Trump wants for profit and Biden wants specifically to limit access to compute because AI may be risky.

Likewise hot wars right where the chips are being made, say if North Korea starts fighting South Korea again, or if China goes for Taiwan.

Re: What happens if we remove 50 percent of Llama?

#114

Post by bob1029 »

drdaeman wrote: post #92

I'm curious - what happens if one prunes the halved model again (if that's possible with the same method), would it start losing accuracy?

At some point you will hit the interpolation threshold and your model will overfit perfectly to the training set.

The gargantuan # of parameters is what buys you the generalization properties that everyone is interested in. A very reduced model may still look & sound competent on the surface, but extensive use by domain experts would quickly highlight the cost of this.

Re: What happens if we remove 50 percent of Llama?

#115

Post by furrypony »

bravura wrote: post #84
celltalk wrote: post #46

All of these smaller model paradigm suggests that we need to incorporate pruning into model training. Neat was one of my favorite algorithms of all time. Same thing with BitNet models which keep showing the information you need is not that much for neural networks. And again, it is same with us, we use much less energy than a regular network so there seems to be immense waste of energy training these models. My intiu…

I seem to recall that there a recent theory paper that got a best paper award, but can't find it. If I remember correctly, their counter-intuitive result was that big overparameterized models could learn more efficiently, and were less likely to get trapped in poor regions of the optimization space. [This is also similar to how introducing multimodal training gives an escape hatch to get out of tricky regions.] So wi…

"Train large, then compress"

Re: What happens if we remove 50 percent of Llama?

#116

Post by dhash »

devsda wrote: post #6

I don't understand LLMs enough to know if this is a silly question or not. Is it possible to build domain specific smaller models and merge/combine them at query/run time to give better response or performance instead of one large all knowing model that learns everything ?

Taking a further step back from LLM’s, this is called portfolio / ensemble techniques in the literature.

A common practice in more formal domains is to have a portfolio of solvers and race them, allowing for the first (provably correct) solver to “win”

In less formal domains, adding/removing nodes/trees in an online manner is part of the deployment process for random forests.

Re: What happens if we remove 50 percent of Llama?

#117

Post by nomel »

celltalk wrote: post #46

All of these smaller model paradigm suggests that we need to incorporate pruning into model training. Neat was one of my favorite algorithms of all time. Same thing with BitNet models which keep showing the information you need is not that much for neural networks. And again, it is same with us, we use much less energy than a regular network so there seems to be immense waste of energy training these models. My intiu…

In mice, ~30% of neurons are silent [1]. Neuralink team is finding that most are silent, where they probe [2]:

> Also, most of them are silent. They don’t really do much. Or their activities are… You have to hit it with just the right set of stimulus.

> ... When you place these electrodes, again, within this hundred micron volume, you have 40 or so neurons. Why do you not see 40 neurons? Why do you see only a handful? What is happening there?

(Yes, I understand LLM aren't brains.)

[1] https://news.mit.edu/2022/silent-synapses-brain-1130

[2] https://youtube.com/watch?v=Kbk9BiPhm7o&t=7056

Re: What happens if we remove 50 percent of Llama?

#118

Post by mewpmewp2 »

visarga wrote: post #107
celltalk wrote: post #46

All of these smaller model paradigm suggests that we need to incorporate pruning into model training. Neat was one of my favorite algorithms of all time. Same thing with BitNet models which keep showing the information you need is not that much for neural networks. And again, it is same with us, we use much less energy than a regular network so there seems to be immense waste of energy training these models. My intiu…

The fact that you can prune a model will not make it smarter, the wall still stands. I think what explains the wall is the fact that we can't scale organic data exponentially, and we have already covered the most useful types. Going forward we will accumulate truly useful data at a linear growing rate. This fundamentally breaks the scaling game. If your model and compute expand exponentially but your training data on…

Don't we still have a lot of video, and other non text real world data to go with? Feels like a possible potential break from there.

Re: What happens if we remove 50 percent of Llama?

#119

Post by spiritplumber »

xvector wrote: post #32
kranner wrote: post #7

Earlier quoted context omitted.

There was that 2007 case of the French man missing 90% of his brain and still quite functional: https://www.cbc.ca/radio/asithappens/as-it-happens-thursday-...

This is really interesting from the perspective of gradual replacement/mind uploading: what is the absolute minimum portion of the brain that we would have to target? Understanding this could probably make the problem easier by some factor (but not "easy" in any sense.)

https://www.badspacecomics.com/post/dementia-ward Obligatory

Re: What happens if we remove 50 percent of Llama?

#120

Post by moralestapia »

moralestapia wrote: post #85
jdietrich wrote: post #82

Earlier quoted context omitted.

I just don't see how anyone can see a study comparing the reasoning abilities of various LLMs, see that large LLMs have better reasoning abilities and conclude that LLMs can't reason. LLMs don't have human-like reasoning abilities, but it's just obviously true that they have some capacity for reasoning; that ability seems to scale roughly linearly with model size and training FLOPs.

Yes, but is human-reasoning on the same spectrum as LLM-reasoning? Meaning that only scale will turn the latter into the former? No definitive answer yet, but my bet is on no.

Lol, imagine being downvoted for asking a couple of questions.

Still 5k points to go, though! :D

Post reply on HN