Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#91

Post by knodi123 »

Galaxeblaffer wrote: post #29
knodi123 wrote: post #17

Earlier quoted context omitted.

> I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. I have some theories that this isn't necessary. 1.) Just because the brain is a general-purpose machine great at doing lots of things, doesn't mean it's great at each of those things. Like when two people are playing catch, and one of them sees the first fragments of a parabola and estimates where the ball is going to land-…

I'm also in the camp that believes that we won't reach agi without significantly more compute. I think that consciousness is an emergent property, just like i think life itself is an emergent property, Both need a certain set elements/systems to work. The secret sauce may look simple when we recreate the right conditions, but it's not going to be something that makes it possible without the right hardware so to speak…

But, take for instance, sight. Sighted people use a large percentage of their brain to process visual input. People born congenitally blind are no smarter or dumber for their brains not having to process that input- so clearly that's not the secret sauce.

I'm not convinced consciousness is emergent, I don't really have an opinion on _that_- but I'm > 50% convinced that consciousness itself doesn't require a neural network as large as a human brain's.

Re: Advances in semiconductors are feeding the AI boom

#92

Post by wslh »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Neurons are not just linear algebra pieces by the way. This is why there is bibliography that talks about the complexity of a single neuron. So, comparing at the unit level at this point is apples vs. oranges.

But, yes, the brain continues to be a surprising machine and ML accomplishements are amazing for that machine.

Re: Advances in semiconductors are feeding the AI boom

#93

Post by nuancebydefault »

layer8 wrote: post #80
fsckboy wrote: post #67

Earlier quoted context omitted.

> the current in a CPU transistor is in the milliampere range ? you sure about that? in a single transistor? over what time period, more than nanoseconds? milliamps is huge, and there are millions of transistors on a single chip these days, and with voltage drops of ... 3V? .7V? you're talking major power. FETs should be operating on field more than flow, though there is some capacitive charge/discharge.

You are right, I mixed this up. If you take a CPU running at 100 W with 10 billion transistors (not quite realistically assumed to all be wired in parallel) at 1 V, you would get an average of 0.01 microamps. So the factor would reduce to roughly 10^5.

Wait a minute, a lot of those transistors are switching the same currents since they are in series. Also, FETs only draw most current while switching, so in between switches there's almost no flow of electrons. So in fact you cannot calculate things that way.

Re: Advances in semiconductors are feeding the AI boom

#94

Post by edward28 »

IshKebab wrote: post #52
HarHarVeryFunny wrote: post #19

We already have a 4 trillion transistor "GPU" in the Cerebras WSE-3 (wafer-scale engine), used in Cerebras' data centers. https://www.youtube.com/watch?v=f4Dly8I8lMY

Yeah that doesn't really count. It's the equivalent to like 20 GPUs and costs 200x as much.

It's meant to compete with nvidia's DGX systems with 8 GPUs per node.

Re: Advances in semiconductors are feeding the AI boom

#95

Post by anonymousDan »

I hear a lot about the energy efficiency of animal brains in comparison to e.g. GPUs. However, as far as I can tell most of the numbers reported are for adult brains, which effectively have been sparsified over time. Does anyone know of how the picture changes if we consider baby animal brains, which as I understand it have much denser connectivity and higher energy consumption than adult brains?

Re: Advances in semiconductors are feeding the AI boom

#96

Post by orbital-decay »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Apples to oranges. Gate count indicates nothing when the architectures are nothing alike.

Brain is a spiking network with mutable connectivity, mostly asynchronous. Only the active path is spending energy at a single moment in time, and "compute" is tightly coupled with memory to the point of being indistinguishable. No need to move data anywhere.

In contrast, GPUs/TPUs are clocked and run fully connected networks, they have to iterate over humongous data arrays every time. Memory is decoupled from compute due to the semiconductor process differences between the two. As a result, they waste a huge amount of energy just moving data back and forth.

Fundamental advancements in SNNs are also required, it's not just about the transistors.

Re: Advances in semiconductors are feeding the AI boom

#97

Post by goatlover »

TRDRVR wrote: post #47
layer8 wrote: post #40

Earlier quoted context omitted.

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

Thank you for this - the name neural networks has made a whole generation of people forget that they have an endocrine system. We know things like sleep, hunger, fear, and stress all impact how we think, yet people want to still build this mental model that synapses are just dot products that either reach an activation threshold or don't.

There have been people on HN arguing that the human brain is a biological LLM, because they can't think of any other way it could work, as if we evolved to generate the next token, instead of fitness as organisms in the real world. Where things like eating, sleeping, shelter, avoiding danger, social bonds, reproduction and child rearing are important. Things that require a body.

Re: Advances in semiconductors are feeding the AI boom

#98

Post by layer8 »

nuancebydefault wrote: post #93
layer8 wrote: post #80

Earlier quoted context omitted.

You are right, I mixed this up. If you take a CPU running at 100 W with 10 billion transistors (not quite realistically assumed to all be wired in parallel) at 1 V, you would get an average of 0.01 microamps. So the factor would reduce to roughly 10^5.

Wait a minute, a lot of those transistors are switching the same currents since they are in series. Also, FETs only draw most current while switching, so in between switches there's almost no flow of electrons. So in fact you cannot calculate things that way.

Yes, as I said the parallel assumption is not quite realistic, and the number is an average, covering all states a transistor may be in. So it amounts to a rough lower bound for when a transistor is switching.

Re: Advances in semiconductors are feeding the AI boom

#99

Post by goatlover »

luyu_wu wrote: post #49
zer00eyz wrote: post #46

Earlier quoted context omitted.

>> 100 trillion synapses (very roughly analogous to transistors?) Not even remotely comparable * its unlikely synapses are binary. Candidly they probably serve more than one purpose. * transistor count is a bad proxy for other reasons. A pipeline to do floats are not going to be useful for fetch from memory. "Where" the density lies is important. * Power: On this front transistors are a joke. * The brain is clockless…

Transistor power is really not a joke. Synapses would take far far FAR more power at close to similar frequencies. Biological neurons are incredibly inefficient.

Biology is incredibly efficient at what it does well though. Thus only 20 watts of brain energy to coordinate everything we do. We didn't evolve to be mentats.

Re: Advances in semiconductors are feeding the AI boom

#100

Post by kungito »

goatlover wrote: post #97
TRDRVR wrote: post #47

Earlier quoted context omitted.

Thank you for this - the name neural networks has made a whole generation of people forget that they have an endocrine system. We know things like sleep, hunger, fear, and stress all impact how we think, yet people want to still build this mental model that synapses are just dot products that either reach an activation threshold or don't.

There have been people on HN arguing that the human brain is a biological LLM, because they can't think of any other way it could work, as if we evolved to generate the next token, instead of fitness as organisms in the real world. Where things like eating, sleeping, shelter, avoiding danger, social bonds, reproduction and child rearing are important. Things that require a body.

I'm one of those people. To me those things only sounded like a different prompt. Priorities set for the llm
Post reply on HN