Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#71

Post by jvanderbot »

layer8 wrote: post #40
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

A simple discretization of the various levels of signal at each input/output, a discretization to handle time-of-propagation (which is almost surely part of the computation just because it _can be_ and nature probably hijacks all mechanisms), and a further discretization to handle the various serum levels in the brain, which are either inputs, outputs, or probably both.

Just add a factor 2^D transistors for each original "brain transistor" and re-run your hardware. Hope field effects don't count, and cross your fingers that neurons are idempotent!

Easy! /s

Modelling an analog system in digital will always have a combinatorial curse of dimensionality. Modelling a biological system is so insanely complex I can't even begin to think about it.

Re: Advances in semiconductors are feeding the AI boom

#72

Post by haltIncomplete »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Why are static 3D cells going to get us there when other ideas have not? Is it needed to replicate “arbitrary” academic ideas of consciousness (despite our best efforts our models are always approximation) to make a useful machine?

“Living things” are not static designs off the drafters table. They’ll never intelligent from their own curiosity, but from ours and the rules we embed. No matter how hard we push puerile hallucinations embedded by Star Trek. It’s still a computer and human agency does not have to bend to it.

Re: Advances in semiconductors are feeding the AI boom

#73

Post by HarHarVeryFunny »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

> The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz).

Partly, but also because the brain has an asynchronous data-flow design, while the GPU is synchronous, and as you say clocked at a very high frequency.

In a clocked design the clock signal needs to be routed to every element on the chip which requires a lot of power, the more so the higher the frequency is. It's a bit like the amount of energy used doing "battle ropes" at the gym. The heavier the ropes (cf more gates the clock is connected to), the more power it takes to move them, and the faster you want to move them (cf faster clock frequency) the more power it takes.

In a data-flow design, like the brain, there is no clock. Each neuron fires, or not, independent of what other neurons are doing, based on their own individual inputs. If the inputs are changing (i.e. receiving signal spikes from attached neurons), then at some threshold of spike accumulation the neuron will fire (expending energy). If the inputs are not changing, or at a level below threshold, then the neuron will not fire.

To consider the difference, imagine our visual cortex if we're looking at a seagull flying across a blue sky. The seagull represents a tiny part of the visual field, and is the only part that is moving/changing, so there are only a few neurons who's inputs are changing and which themselves will therefore fire and expend energy. The blue sky comprising the rest of the visual field is not changing and we therefore don't expend any energy reprocessing it over and over.

In contrast, if you fed a video (frame by frame) of that same visual scene into a CNN being processed on a GPU, then it does not distinguish between what is changing or not, so 95% of the energy processing each frame will be wasted, and this will be repeated frame by frame as long as we're looking at that scene!

Re: Advances in semiconductors are feeding the AI boom

#74

Post by omikun »

layer8 wrote: post #40
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

A single precision flop is in the order of pJ. [1] A transistor would be much less.

[1] https://arxiv.org/pdf/1809.09206.pdf

Re: Advances in semiconductors are feeding the AI boom

#76

Post by doug_durham »

barelyauser wrote: post #64
vasco wrote: post #55

Earlier quoted context omitted.

We also lose a lot when building computers due to the fact we have to convert the analog world into digital representations. A neural analog computer would be more efficient I think, and due to the non-deterministic nature of AI would probably suit the task as well.

Non-deterministic means random. AI or natural I is not random. Analog suffers immensely from noise and it is the reason the brain has such a large number of neurons, part to deal with noise and part to deal with losing some neurons along the way.

Non-deterministic doesn't mean random. Random means random. Non-deterministic means that specific inputs don't generate the same outputs. It says nothing about the distribution of the output values. A chaotic system isn't random, its just non-deterministic.

Re: Advances in semiconductors are feeding the AI boom

#77

Post by kazinator »

Hardware, or at least purely computational hardware, will never get the same accolades as the software that actually makes it do something.

Interface hardware, being is perceptible to the senses, gets credit over software.

E.g. when people experience a vivid, sharp high resolution display, they attribute all its good properties to the hardware, even if there is some software involved in improving the visuals, like making fonts look better and whatnot.

If a mouse works nicely, people attribute it to the hardware, not the drivers.

If you work in hardware, and crave the appreciation, make something that people look at, hear, or hold in their hands, not something that crunches away in a closet.

Re: Advances in semiconductors are feeding the AI boom

#78

Post by [deleted] »

mrtksn wrote: post #45
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

IMHO the breakthrough will come with an analog computer. The current systems simulate stuff through computation using switches. It’s like real sand vs sand simulator in the browser. One spins your fans and drains your battery when showing you 1000 particles acting like sand, the other just obeys laws of physics locally per particle and can do millions of particles much more accurately and with very slight increase in…

[deleted]

Re: Advances in semiconductors are feeding the AI boom

#79

Post by taktoa »

HarHarVeryFunny wrote: post #73
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

> The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). Partly, but also because the brain has an asynchronous data-flow design, while the GPU is synchronous, and as you say clocked at a very high frequency. In a clocked design the clock signal needs to be routed to every element on the chip which requires a lot of power, the more so the higher the fre…

> In a clocked design the clock signal needs to be routed to every element on the chip which requires a lot of power, the more so the higher the frequency is.

Clock only needs to be distributed to sequential components like flip flops or SRAMs. The number of clock distribution wire-millimeters in typical chip is dwarfed by the number of data wire-millimeters, and if a neural network is well trained and quantized activations should be random, so number of transitions per clock should be 0.5 (as opposed to 1 for clock wires), meaning that power can't be dominated by clock. The flops that prevent clock skew are a small % of area, so I don't think those can tip the scales either. On the other hand, in asynchronous digital logic you need to have valid bit calculation on every single piece of logic, which seems like a pretty huge overhead to me.

Re: Advances in semiconductors are feeding the AI boom

#80

Post by layer8 »

fsckboy wrote: post #67
layer8 wrote: post #40

Earlier quoted context omitted.

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

> the current in a CPU transistor is in the milliampere range ? you sure about that? in a single transistor? over what time period, more than nanoseconds? milliamps is huge, and there are millions of transistors on a single chip these days, and with voltage drops of ... 3V? .7V? you're talking major power. FETs should be operating on field more than flow, though there is some capacitive charge/discharge.

You are right, I mixed this up. If you take a CPU running at 100 W with 10 billion transistors (not quite realistically assumed to all be wired in parallel) at 1 V, you would get an average of 0.01 microamps. So the factor would reduce to roughly 10^5.
Post reply on HN