Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

Post reply on HN
Print view View original

41–50 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#41

Post by marcosdumay »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

On the power usage, the difference is that those synapses are almost always in stand-by. The equivalent would be a CMOS circuit with a clock of minutes.

On the complexity, AFAIK a synapse is way more complex than a transistor. Larger too, if you include its share of the neuron's volume. And yes, the count difference is due to the 3D packing.

Re: Advances in semiconductors are feeding the AI boom

#42

Post by highfrequency »

npalli wrote: post #38
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

No neurons are the equivalent of transistors. Synapses are the equivalent of connections between transistors. The total neurons are about 100 Billion but the connections are 100 Trillion. A neuron can have up to 10,000 synapses, while a transistor can have on average only about 3. Impressive nonetheless on power efficiency.

# synapses should be analogous to # model parameters no? And # model parameters should be linear in # transistors.

Re: Advances in semiconductors are feeding the AI boom

#43

Post by passion__desire »

gamacodre wrote: post #39
passion__desire wrote: post #20

Earlier quoted context omitted.

Is there a comparison of power efficiency of human brain doing 50 digits multiplication vs a multiplier circuit doing it?

I think the problem here would be figuring out how much of the brain's power draw to attribute to the multiplication. A brain is more akin to a motherboard than a single CPU, with all kinds of I/O, internal regulation, and other ancillary stuff going on all the time.

Is the issue then we haven't discovered the magical algorithm run by our brain? If we discover it, then digital circuits will handsomely beat brain.

Re: Advances in semiconductors are feeding the AI boom

#44

Post by marcosdumay »

ramshanker wrote: post #4

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

I doubt it. Digital just scales better.

Our brain has a pretty bounded need of scaling, but once we create some computer equivalent, it would be very counterproductive to make it useless for larger problems for a small gain on smaller ones.

Re: Advances in semiconductors are feeding the AI boom

#45

Post by mrtksn »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

IMHO the breakthrough will come with an analog computer.

The current systems simulate stuff through computation using switches.

It’s like real sand vs sand simulator in the browser. One spins your fans and drains your battery when showing you 1000 particles acting like sand, the other just obeys laws of physics locally per particle and can do millions of particles much more accurately and with very slight increase in temperature.

Of course, analog computations are much less controllable but in this case that’s not a deal breaker.

Re: Advances in semiconductors are feeding the AI boom

#46

Post by zer00eyz »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

>> 100 trillion synapses (very roughly analogous to transistors?)

Not even remotely comparable

* its unlikely synapses are binary. Candidly they probably serve more than one purpose.

* transistor count is a bad proxy for other reasons. A pipeline to do floats are not going to be useful for fetch from memory. "Where" the density lies is important.

* Power: On this front transistors are a joke.

* The brain is clockless, and analog... frequency is an interesting metric

Binary systems are going to be bad at simulating complex processes. LLM's are a simulation of intelligence, like forecasting is a simulation of weather. Lorenz shows us why simulation of weather has limits, there isnt some magical math that will change those rules for ML to make the leap to "AGI"

Re: Advances in semiconductors are feeding the AI boom

#47

Post by TRDRVR »

layer8 wrote: post #40
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

Thank you for this - the name neural networks has made a whole generation of people forget that they have an endocrine system.

We know things like sleep, hunger, fear, and stress all impact how we think, yet people want to still build this mental model that synapses are just dot products that either reach an activation threshold or don't.

Re: Advances in semiconductors are feeding the AI boom

#48

Post by phkahler »

highfrequency wrote: post #42
npalli wrote: post #38

Earlier quoted context omitted.

No neurons are the equivalent of transistors. Synapses are the equivalent of connections between transistors. The total neurons are about 100 Billion but the connections are 100 Trillion. A neuron can have up to 10,000 synapses, while a transistor can have on average only about 3. Impressive nonetheless on power efficiency.

# synapses should be analogous to # model parameters no? And # model parameters should be linear in # transistors.

>> # synapses should be analogous to # model parameters no?

I think they're equivalent to a parameter AND the multiplier. Or in analog terms they'd just be a resistor whose value can be changed. Digital stuff is not a good fit for this.

Re: Advances in semiconductors are feeding the AI boom

#49

Post by luyu_wu »

zer00eyz wrote: post #46
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

>> 100 trillion synapses (very roughly analogous to transistors?) Not even remotely comparable * its unlikely synapses are binary. Candidly they probably serve more than one purpose. * transistor count is a bad proxy for other reasons. A pipeline to do floats are not going to be useful for fetch from memory. "Where" the density lies is important. * Power: On this front transistors are a joke. * The brain is clockless…

Transistor power is really not a joke. Synapses would take far far FAR more power at close to similar frequencies. Biological neurons are incredibly inefficient.

Re: Advances in semiconductors are feeding the AI boom

#50

Post by phkahler »

If we are just after AI now we should drop the GPU concept and call it what it is - Matrix multipliers. With that framing, we can move to in-memory compute so the data doesn't have to be moving around so much. Memory chips could have lines of MAC units at some intervals and something to compute the nonlinear functions after summing. Fixed sizes could be implemented in hardware and software would spread larger computations over a number of tiles. If this were standardized we might see it end up as a modest premium price on memory chips. nVidia step aside it's going to be Micron, Hynix and friends revolutionizing AI.
Post reply on HN