61–70 of 139 posts
Earlier quoted context omitted.
AMD Radeon series ≥6800 & ≥7800 have 16GB VRAM too.
Even RX 7600 XT has 16GB
Nonetheless, great info. Sounds like it might be the budget inference king!
Earlier quoted context omitted.
> This would give LLM functioning autism Functioning autism hardly equals low intellect. Half the people of this forum (at least) are functioning autists.
No, but it's also true that almost 40% of autists have intellectual disabilities: https://www.cdc.gov/mmwr/volumes/72/ss/ss7202a1.htm That said, the parent comment is just silly and wrong.
After reading the article it seems to me that this is more like synaptic pruning where weak connections between neurons are eliminated in order to increase the efficiency of the neurons. Interesting to see that this also works for LLMs. https://en.wikipedia.org/wiki/Synaptic_pruning
After reading the article it seems to me that this is more like synaptic pruning where weak connections between neurons are eliminated in order to increase the efficiency of the neurons. Interesting to see that this also works for LLMs. https://en.wikipedia.org/wiki/Synaptic_pruning
The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pruning in LLMs is no exception.
During the learning stage we want input from every variable so that we are sure that we don't omit a variable that turns out to be essential for the calculation. However in any calculation a human does 99.9999% of variables are irrelevant (e.g. what day of the week it is, am I sleepy, etc), so of course the brain wouldn't use resources to keep connections that aren't relevant to a given function. Imagine what a liability it would be if we have had excessive direct connections from our visual processing system to the piece of our brain that controls heartrate.
Earlier quoted context omitted.
> Nobody understands how or why the router chooses the expert. It’s so early. Nobody understand how LLM works either. Is LLM as "early" as MoE ?
LLMs are really well understood, what do you mean? You can see the precise activations and token probabilities for every next token. You can abliterate the network however you'd like to suppress or excite concepts of your choosing.
Earlier quoted context omitted.
The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pruning in LLMs is no exception.
Really? It seems obvious to me. During the learning stage we want input from every variable so that we are sure that we don't omit a variable that turns out to be essential for the calculation. However in any calculation a human does 99.9999% of variables are irrelevant (e.g. what day of the week it is, am I sleepy, etc), so of course the brain wouldn't use resources to keep connections that aren't relevant to a give…
Earlier quoted context omitted.
No, but it's also true that almost 40% of autists have intellectual disabilities: https://www.cdc.gov/mmwr/volumes/72/ss/ss7202a1.htm That said, the parent comment is just silly and wrong.
What i want to mean is difference between 100% fine person vs Functioning Autist. Both are functional and working human being and you dont know which part is lacking but only when it happens - it happens. Make sense?
Surprising that the retained accuracy is so high after removing 1/2 of parameters. Does this help with being able to run inference on low-end GPUs?