Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

Post reply on HN
Print view View original

101–110 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#101

Post by Traubenfuchs »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Neurons and synapses have more than 100 different neurotransmitters and their many receptors, there is reuptake and destructive encyme activity and they are connected up to many thousands of their peer. Every single neuron is a dizzyingly complex machine, employing countless sub machines and processes.

You can not reasonably compare this to model parameters or transistors at all.

Re: Advances in semiconductors are feeding the AI boom

#102

Post by ozim »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Nature was “building” this stuff for eons. I feel pretty good about our progress in still less than a 100 years.

Re: Advances in semiconductors are feeding the AI boom

#103

Post by TRDRVR »

kungito wrote: post #100
goatlover wrote: post #97

Earlier quoted context omitted.

There have been people on HN arguing that the human brain is a biological LLM, because they can't think of any other way it could work, as if we evolved to generate the next token, instead of fitness as organisms in the real world. Where things like eating, sleeping, shelter, avoiding danger, social bonds, reproduction and child rearing are important. Things that require a body.

I'm one of those people. To me those things only sounded like a different prompt. Priorities set for the llm

Here's the hubris of thinking that way:

I would imagine the baseline assumption of your thinking is that things like sleep and emotions are a 'bug' in terms of cognition (or at the very least, 'prompts' that are optional).

Said differently, the assumption is that with the right engineer, you could reach human-parity cognition with a model that doesn't sleep or feel emotions (after all what's the point of an LLM if it gets tired and doesn't want to answer your questions sometimes? Or even worse knowingly deceives you because it is mad at you or prejudiced against you).

The problem with that assumption is that as far as we can tell, every being with even the slightest amount of cognition sleeps in some form and has something akin to emotional states. As far as we can prove, sleep and emotions are necessary preconditions to cognition.

A worldview where the 'good' parts of the brain (reasoning and logic) are replicated in LLM but the 'bad' parts (sleep, hunger, emotions, etc.) are not is likely an incomplete model.

Re: Advances in semiconductors are feeding the AI boom

#104

Post by xcv123 »

AnimalMuppet wrote: post #11
tzm wrote: post #2

Cerebras WSE-3 contains 4 trillion transistors and 8 exaflops per sec, 20 PB bandwidth. 62 times the cores of an H100.. 900,000. I wonder if the WSE-3 can compete on price / performance though. Interesting times!

9 trillion flops per core? That's... mind-boggling. Is that real? And, 9 trillion flops per core in 4.4 million transistors per core. That sounds a bit too good to be true.

No the real performance is 125 petaflops at FP16. That is 138 gigaflops per core.

You would need 64 of these to get 8 exaflops.

https://www.tomshardware.com/tech-industry/artificial-intell...

Re: Advances in semiconductors are feeding the AI boom

#105

Post by xcv123 »

tzm wrote: post #2

Cerebras WSE-3 contains 4 trillion transistors and 8 exaflops per sec, 20 PB bandwidth. 62 times the cores of an H100.. 900,000. I wonder if the WSE-3 can compete on price / performance though. Interesting times!

> 8 exaflops per sec

Your number is off by 64x.

It can do 125 petaflops at FP16

https://www.tomshardware.com/tech-industry/artificial-intell...

Re: Advances in semiconductors are feeding the AI boom

#106

Post by goatlover »

kungito wrote: post #100
goatlover wrote: post #97

Earlier quoted context omitted.

There have been people on HN arguing that the human brain is a biological LLM, because they can't think of any other way it could work, as if we evolved to generate the next token, instead of fitness as organisms in the real world. Where things like eating, sleeping, shelter, avoiding danger, social bonds, reproduction and child rearing are important. Things that require a body.

I'm one of those people. To me those things only sounded like a different prompt. Priorities set for the llm

Isn’t that taken the analogy too literally? You’re saying nature is promoting humans to generate the next token to be outputted? What about all the other organisms that don’t have language? How do you distinguish nature prompts from nature training datasets? What makes you think nature is tokenized? What makes you think language generation is fundamental to biology?

Re: Advances in semiconductors are feeding the AI boom

#107

Post by 0x000xca0xfe »

highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

It has some interesting implications: If we manage to replicate a human-like brain in silicon, it could think multiple million times faster than the biological equivalent. So even if it needs a lot of energy to run, it could "out-think" us by orders of magnitude and would have crazy reaction times. Just imagine a sentient being with the human mental equivalent of millions of years in one year. Crazy.

But we know very little about how biological brains actually work and very few connectomes have been fully mapped as of yet. We still cannot fully explain how C. elegans with 302 neurons "thinks".

Also every neuron contains the entire genome, which is hundreds of millions to billions of base pairs depending on the animal and we also don't really know how important it is to the function of the brain.

So the "state" of biological brains is potentially utterly gigantic even for simple animals.

Re: Advances in semiconductors are feeding the AI boom

#108

Post by eli_gottlieb »

layer8 wrote: post #40
highfrequency wrote: post #26

Wild that the human brain can squeeze in 100 trillion synapses ( very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz). My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integr…

Ignoring for the moment that transistors and synapses are very different in their function, the current in a CPU transistor is in the milliampere range, whereas in the ion channels of a synapse it is in the picoampere range. The voltage differs by roughly a factor of ten. So the wattage differs by a factor of 10^10. One important reason for the difference in current is that transistors need to reliably switch between…

Synapses are very analog, but then the neuronal soma (cell body) and axon are pretty pulsatile again. A spike is fired or it isn't!

Re: Advances in semiconductors are feeding the AI boom

#109

Post by eli_gottlieb »

gary_0 wrote: post #86
__loam wrote: post #70

Earlier quoted context omitted.

People will spout off about how machine learning is based on the brain while having no idea how the brain works.

It is based on the brain, but only in the loosest possible terms; ML is a cargo cult of biology. It's kind of surprising that it works at all.

Honestly once you understand maximum-likelihood estimation, empirical risk minimization, automatic differentiation, and stochastic gradient descent, it's not that much of a surprise it works.

Re: Advances in semiconductors are feeding the AI boom

#110

Post by kelseyfrog »

TRDRVR wrote: post #103
kungito wrote: post #100

Earlier quoted context omitted.

I'm one of those people. To me those things only sounded like a different prompt. Priorities set for the llm

Here's the hubris of thinking that way: I would imagine the baseline assumption of your thinking is that things like sleep and emotions are a 'bug' in terms of cognition (or at the very least, 'prompts' that are optional). Said differently, the assumption is that with the right engineer, you could reach human-parity cognition with a model that doesn't sleep or feel emotions (after all what's the point of an LLM if it…

Do airplanes need sleep because they fly like birds who also require sleep?
Post reply on HN