Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

Post reply on HN
Print view View original

111–120 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#111

Post by ksec »

npalli wrote: post #38
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

No neurons are the equivalent of transistors. Synapses are the equivalent of connections between transistors. The total neurons are about 100 Billion but the connections are 100 Trillion. A neuron can have up to 10,000 synapses, while a transistor can have on average only about 3. Impressive nonetheless on power efficiency.

So the actual equation should be 100 billion times 100 Trillion times clock speed? ( I have zero ideas about how our brains works )

Even we have 10 times higher clock speed, a trillion transistor GPU is still very slow comparatively speaking ?

Surely memory has to come in somewhere?

Re: Advances in semiconductors are feeding the AI boom

#112

Post by Terr_ »

goatlover wrote: post #97
TRDRVR wrote: post #47

Earlier quoted context omitted.

Thank you for this - the name neural networks has made a whole generation of people forget that they have an endocrine system. We know things like sleep, hunger, fear, and stress all impact how we think, yet people want to still build this mental model that synapses are just dot products that either reach an activation threshold or don't.

There have been people on HN arguing that the human brain is a biological LLM, because they can't think of any other way it could work, as if we evolved to generate the next token, instead of fitness as organisms in the real world. Where things like eating, sleeping, shelter, avoiding danger, social bonds, reproduction and child rearing are important. Things that require a body.

It's also frustrating because LLMs aren't even the only kind of AI/ML out there, they're just the kind currently getting investment and headlines.

Re: Advances in semiconductors are feeding the AI boom

#113

Post by Terr_ »

Borg3 wrote: post #90
gary_0 wrote: post #86

Earlier quoted context omitted.

It is based on the brain, but only in the loosest possible terms; ML is a cargo cult of biology. It's kind of surprising that it works at all.

It works because well, its actually pretty primitive at its core. Whole learning process is actually pretty brutal. Doing millions of interations w/ random (and semi-random) adjustments.

I think I've fallen into the "it's just a very fancy kind of lossy compression" camp.

Re: Advances in semiconductors are feeding the AI boom

#114

Post by ksec »

DesiLurker wrote: post #25
HarHarVeryFunny wrote: post #19

We already have a 4 trillion transistor "GPU" in the Cerebras WSE-3 (wafer-scale engine), used in Cerebras' data centers. https://www.youtube.com/watch?v=f4Dly8I8lMY

how many regular sized GPU (say 4080 or nvidia L4) dies can be cut out from a full-sized wafer? I suppose thats what OP means by reaching integration density of 1T GPU.

At close to reticle limit of around ~810mm2, you could fit around 65 chips.

Re: Advances in semiconductors are feeding the AI boom

#115

Post by goatlover »

kelseyfrog wrote: post #110
TRDRVR wrote: post #103

Earlier quoted context omitted.

Here's the hubris of thinking that way: I would imagine the baseline assumption of your thinking is that things like sleep and emotions are a 'bug' in terms of cognition (or at the very least, 'prompts' that are optional). Said differently, the assumption is that with the right engineer, you could reach human-parity cognition with a model that doesn't sleep or feel emotions (after all what's the point of an LLM if it…

Do airplanes need sleep because they fly like birds who also require sleep?

Birds flap their wings and maneuver differently. They don't fly the same way.

Re: Advances in semiconductors are feeding the AI boom

#116

Post by vasco »

barelyauser wrote: post #64
vasco wrote: post #55

Earlier quoted context omitted.

We also lose a lot when building computers due to the fact we have to convert the analog world into digital representations. A neural analog computer would be more efficient I think, and due to the non-deterministic nature of AI would probably suit the task as well.

Non-deterministic means random. AI or natural I is not random. Analog suffers immensely from noise and it is the reason the brain has such a large number of neurons, part to deal with noise and part to deal with losing some neurons along the way.

Analog suffers from noise? What do you think you get when you convert analog to digital? Way more loss of signal than whatever inbuilt noise the analog signal had in the first place.

Re: Advances in semiconductors are feeding the AI boom

#117

Post by IshKebab »

edward28 wrote: post #94
IshKebab wrote: post #52

Earlier quoted context omitted.

Yeah that doesn't really count. It's the equivalent to like 20 GPUs and costs 200x as much.

It's meant to compete with nvidia's DGX systems with 8 GPUs per node.

Yes exactly. It's not comparable to one GPU.

Re: Advances in semiconductors are feeding the AI boom

#118

Post by sorokod »

kelseyfrog wrote: post #110
TRDRVR wrote: post #103

Earlier quoted context omitted.

Here's the hubris of thinking that way: I would imagine the baseline assumption of your thinking is that things like sleep and emotions are a 'bug' in terms of cognition (or at the very least, 'prompts' that are optional). Said differently, the assumption is that with the right engineer, you could reach human-parity cognition with a model that doesn't sleep or feel emotions (after all what's the point of an LLM if it…

Do airplanes need sleep because they fly like birds who also require sleep?

They do need scheduled maintenance.

Re: Advances in semiconductors are feeding the AI boom

#119

Post by HarHarVeryFunny »

IshKebab wrote: post #52
HarHarVeryFunny wrote: post #19

We already have a 4 trillion transistor "GPU" in the Cerebras WSE-3 (wafer-scale engine), used in Cerebras' data centers. https://www.youtube.com/watch?v=f4Dly8I8lMY

Yeah that doesn't really count. It's the equivalent to like 20 GPUs and costs 200x as much.

Well, yeah, that's what a few trillion transistors look like with today's tech. No doubt it will get smaller/cheaper in the future (although EUV is expensive, so maybe prices won't keep dropping as much as in the past). The point is that even with today's tech a (4) trillion transistor chip can already be built.

Re: Advances in semiconductors are feeding the AI boom

#120

Post by TRDRVR »

kelseyfrog wrote: post #110
TRDRVR wrote: post #103

Earlier quoted context omitted.

Here's the hubris of thinking that way: I would imagine the baseline assumption of your thinking is that things like sleep and emotions are a 'bug' in terms of cognition (or at the very least, 'prompts' that are optional). Said differently, the assumption is that with the right engineer, you could reach human-parity cognition with a model that doesn't sleep or feel emotions (after all what's the point of an LLM if it…

Do airplanes need sleep because they fly like birds who also require sleep?

Ah a very fun 'snippy' question that just proves my point further. Thank you.

No airplanes do not sleep. That's part of why their flying is fundamentally different than birds'.

You'll likely also notice that birds flap their wings while planes use jet engines and fixed wings.

My entire point is that it is foolish to imagine airplanes as mechanical birds, since they are in fact completely different and require their own mental models to understand.

This is analogous to LLMs. They do something completely different than what our brains do and require their own mental models in order to understand them completely.

Post reply on HN