Live data from Hacker News

Averaging is a convenient fiction of neuroscience

thetransmitter.org

91–100 of 110 posts

Re: Averaging is a convenient fiction of neuroscience

#91
post #84
post #63

Earlier quoted context omitted.

An elephant brain has 3 times as many neurons as a human. They are pretty smart animals but so are dogs who have way less neurons. The point here being that the number of neurons is just one of the many factors that determines intelligence (general or not)

Even among humans with roughly similar neuron counts, there are notable examples of individuals displaying extreme stupidity.

To further complicate matters, some forms of stupidity are not the lack of intelligence but the consequence of other cognitive processes that may (or would) otherwise useful in social animals because they reinforce group bonds and have a role in group identity definition. Not all those processes are well adapted to the modern society.

Re: Averaging is a convenient fiction of neuroscience

#92
post #80

Earlier quoted context omitted.

Yeah it's a common mistake, but this is like intro to stats stuff. It's not some big secret that if you summarize a distribution with a single number then you've lost information. > I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things. I'd counter that it's easily one of the biggest assistants in understanding things. The Central Limit Theorem in partic…

> this is like intro to stats stuff Amazing how the replies to my initial comment range from angrily calling me an idiot for saying something so obviously and egregiously wrong, to claiming it is so trivial and obvious that I should not have needed to mention it. > The solution isn't less literacy. Indeed, I can see how what I wrote could be taken that way, I also don't think so. The problem is combining an inadequat…

To clarify, I'm not saying that it's trivial or obvious, just that it's fundamental. Just like derivatives aren't trivial or obvious but everyone who has taken Intro to Calculus knows about them.

> "e.g. scientism"

Isn't this just an anti-science term adopted by religious people and postmodernists?

Re: Averaging is a convenient fiction of neuroscience

#93

As a computer scientist, I was blown away the first time my friend explained to me that his research focused on the timing of neuron spikes, not their magnitude. After talking about it for a while I realized that machine learning neural networks are much closer to simple early models of how neuron's work (averages and all), not how neuron's actually signal. Makes sense when you consider how the latest LLM models have…

> the latest LLM models have almost as many parameters as we have neurons I often see this take, but the apt comparison is between parameter and synapse count, not neuron count. You should be counting hidden units rather than weights if you want to compare to neuron counts.

To expand on that, as a point of comparison, a single neuron can have thousands of synaptic connections. So we're still a few orders of magnitude out from modeling NNs that have a degree of connectivity similar to the brain, even though the synapse counts are comparable.

Re: Averaging is a convenient fiction of neuroscience

#94
post #80

Earlier quoted context omitted.

> this is like intro to stats stuff Amazing how the replies to my initial comment range from angrily calling me an idiot for saying something so obviously and egregiously wrong, to claiming it is so trivial and obvious that I should not have needed to mention it. > The solution isn't less literacy. Indeed, I can see how what I wrote could be taken that way, I also don't think so. The problem is combining an inadequat…

To clarify, I'm not saying that it's trivial or obvious, just that it's fundamental. Just like derivatives aren't trivial or obvious but everyone who has taken Intro to Calculus knows about them. > "e.g. scientism" Isn't this just an anti-science term adopted by religious people and postmodernists?

> Isn't this just an anti-science term adopted by religious people and postmodernists?

Not at all, as a scientist, I find scientism to be very common and the biggest obstacle to the general public understanding science- more so than religious people and postmodernists which are at least openly anti-science, rather than just confused about what it is. It is a fanatical religious like belief in the correctness, completeness, and finality of "what science has already discovered" and an aggressive dismissal of e.g. things like creative though experiments or anecdotal observations that question current understanding. To people practicing scientism, the regular practice of actual science looks like heresy and pseudo-science and they would get angry if they were present, e.g. when Isaac Newton or Charles Darwin first mentioned their now accepted but once controversial ideas. They will say things like you don't know (and shouldn't think or talk about) anything until it is confirmed in a large study- without considering how scientists come up with the designs and ideas (and funding) for such studies in the first place.

Science in popular media and schools is taught as a bunch of authoritarian facts to memorize and believe without deeply and intuitively understanding, rather than a creative process of trying to deeply understanding and question things for yourself. When I mention something interesting or counter-intuitive I encountered in my professional research, or something interesting I have been thinking about recently- especially among young intelligent "skeptical and science loving" non-scientist programmers on this forum, I am often met with angry derision, because the process of talking about and exploring weird ideas- e.g. the actual how the sausage is made by working scientists, is seen as extremely unacceptable if it conflicts with what they heard in school, media, etc. I’ve mentioned things I’ve personally discovered on here- and been called an idiot and sent the wikipedia link for a field I’ve studied for decades, and is actually citing my older papers and work, by someone that only heard the field existed when they read my comment.

The pseudo-science conspiracy theory people ironically get the creative independent thinking part more right than the "Scientism-ists", but then miss the important second part of actually taking the responsibility to deeply understand for yourself, and critically question things.

There is something very wrong with science education when the people that say they love it, respond negatively when they encounter the real thing- and this can be fixed by redesigning science education. As an academic PI I don’t teach regular classes, but put students directly in the lab working on a new problem nobody has solved before, where they can’t look up the answer, and then mentor them 1 on 1 if they get really stuck. I then have them actually write it up and submit it to a real peer reviewed journal.

Re: Averaging is a convenient fiction of neuroscience

#95
post #63

As a computer scientist, I was blown away the first time my friend explained to me that his research focused on the timing of neuron spikes, not their magnitude. After talking about it for a while I realized that machine learning neural networks are much closer to simple early models of how neuron's work (averages and all), not how neuron's actually signal. Makes sense when you consider how the latest LLM models have…

An elephant brain has 3 times as many neurons as a human. They are pretty smart animals but so are dogs who have way less neurons. The point here being that the number of neurons is just one of the many factors that determines intelligence (general or not)

80% of human neurons are actually in the cerebellum and are not related to consciousness at all

Re: Averaging is a convenient fiction of neuroscience

#96
post #94

Earlier quoted context omitted.

To clarify, I'm not saying that it's trivial or obvious, just that it's fundamental. Just like derivatives aren't trivial or obvious but everyone who has taken Intro to Calculus knows about them. > "e.g. scientism" Isn't this just an anti-science term adopted by religious people and postmodernists?

> Isn't this just an anti-science term adopted by religious people and postmodernists? Not at all, as a scientist, I find scientism to be very common and the biggest obstacle to the general public understanding science- more so than religious people and postmodernists which are at least openly anti-science, rather than just confused about what it is. It is a fanatical religious like belief in the correctness, complet…

Oh okay, thanks for the response. I see more where you're coming from.

I don't think I'd describe what you're talking about as "scientism", it's more just a confusion about what science is.

There are a lot of critiques of so-called "scientism" but they focus on the assertion that there are ways of "knowing" things that are inaccessible to the scientific process. This more or less directly implies the supernatural (since we have to leave naturalism to understand it) or trolling (which is basically what postmodernism is).

Part of what you're running into in this site is that there is a political dislike of education, knowledge, and science in some coding circles. Your original comment brought out a few of them.

Re: Averaging is a convenient fiction of neuroscience

#97

As a computer scientist, I was blown away the first time my friend explained to me that his research focused on the timing of neuron spikes, not their magnitude. After talking about it for a while I realized that machine learning neural networks are much closer to simple early models of how neuron's work (averages and all), not how neuron's actually signal. Makes sense when you consider how the latest LLM models have…

Human neural network build and trim connections constantly [1]. I imagine we will get much closer to AGI if we can update models dynamically, instead of just adding more neurons and more training. After all human didn't need reading billions of articles before writing an average one.

[1] https://en.wikipedia.org/wiki/Neuroplasticity

Re: Averaging is a convenient fiction of neuroscience

#98

Earlier quoted context omitted.

I've heard this argument a million times, but I am very skeptical: where would the reliable data on infant mortality in ancient times come from? (so that it would allow us to compute precise values of average lifespan). All we have from those times are a few bone samples and a few anecdotes preserved in fragments. I'm not saying anything for or against the ~33 years claim, just that I doubt that it comes from a preci…

> All we have from those times are a few bone samples and a few anecdotes preserved in fragments. This is the bit that you're missing: antiquity is generally considered to extend all the way into the 500-600s AD, with the Roman empire dominating its center and 'late antiquity' covering the whole early period of the Byzantine empire. This means that even just in Europe we have extensive documents (e.g. in the Christia…

[deleted]

Re: Averaging is a convenient fiction of neuroscience

#99
post #38

Earlier quoted context omitted.

>Sounds like we mostly agree, save some pedantry. If by pedantry you mean that I'm not ignoring the cause of 70% of the improvement in life expectancy in the last 200 years then sure.

Indeed, this conversation is a good illustration of the damage that Bayesian statistics have done to the "educated" populace. Not that they're inherently bad--it's just a different statistical approach, and it's generally good not to assume a universal background, that everything is normal, etc.--but by telling people it's fine to question statistical conclusions because the distribution might be different, it libera…

Yes, bunch of ignorant beasts we are.

It’s totally reasonable and statistically sound to use statistics from Victorian England as the sole proxy for the pre-modern human race. /s

Re: Averaging is a convenient fiction of neuroscience

#100
Another convenient fiction is that neuronal communication is all spikes, 1's and 0's. In this fiction, neuromodulators are ignored. Glial cells are ignored. The immune system is ignored. The first neurons in the brain that select for different auditory frequencies are ignored - auditory hair cells release a steady stream of vesicles packed with glutamate, and postsynaptic glutamate receptors compute something like a moving average of the glutamate concentration in the synaptic cleft. But it's much, much more complicated than that. Sounds like something is holding back the field, but averaging is a pretty lousy description of what it might be.
Post reply on HN