Live data from Hacker News

Advances in semiconductors are feeding the AI boom

spectrum.ieee.org

21–30 of 124 posts

Re: Advances in semiconductors are feeding the AI boom

#21

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

Analogue circuits can be a real pain in the butt, imagine if integer overflows destroyed the whole ALU, haha.

Transistors are already much smaller than neurons. And of course the brain doesn’t have a clock. And neurons have more complex behavior than single transistors… The whole system is just very different. So, this doesn’t seem like a strategy to get past a boundary, it is more like a suggestion that we give up on the current path and go in a radically different direction. It… isn’t impossible but it seems like a wild change for the field.

If we want something post-cmos, potentially radically more efficient, but still familiar in the sense that it produces digital logic, quantum dot cellular automata with Bennett clocking seems more promising IMO.

Re: Advances in semiconductors are feeding the AI boom

#22
post #8

Yeah I also think that the impact of better hardware generates a visible lead in quality in the models that are released, for example: companies like OpenAI have had access to large quantities of H100 for a few months now and Sora is being presented, something I would not have believed a year ago, I would also believe that Claude 3 models were trained on H100, DBRX was trained on 12T tokens, a big difference compared…

> companies like OpenAI have had access to large quantities of H100 for a few months now and Sora is being presented

From what I could tell from Nvidia's recent presentation, Nvidia works directly with OpenAI to test their next gen hardware. IIRC they had some slides showing the throughput comparisons with Hopper and Blackwell, suggesting they used OpenAI's workload for testing.

H100's have been generally available (not a long waitlist) for only several months, but all the big players had them already 1 year ago.

I agree with you, but I think you might be 1 generation behind.

> OpenAI used H100’s predecessor — NVIDIA A100 GPUs — to train and run ChatGPT, an AI system optimized for dialogue, which has been used by hundreds of millions of people worldwide in record time. OpenAI will be using H100 on its Azure supercomputer to power its continuing AI research.

March 21, 2023 https://nvidianews.nvidia.com/news/nvidia-hopper-gpus-expand...

Re: Advances in semiconductors are feeding the AI boom

#23

For some sense of how far out/close 1 trillion transistors in one GPU is: NVIDIA just announced Blackwell which gets to 208bn transistors on a chip by stitching two dies together into a single GPU. https://www.nvidia.com/en-us/data-center/technologies/blackw... They’re sticking two of them in a board with a Grace CPU in between, then linking 36 of those those boards together in racks with NVLink switches that offer “…

GPUs can still scale up by a LOT. The real limiting parameters are power consumption (which includes cooling), bus speed, production level of foundries.

Think super-[cross-fire/sli].

Economics will probably forbid that. This is a virtual limit which factors in the previous physical limits... in theory.

Re: Advances in semiconductors are feeding the AI boom

#24

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

I work in analog,

1) Noise is an issue as the system gets complex. You can't get away with counting to 1 anymore, all those levels in between matter. 2) Its hard to make an analog computer reconfigurable. 3) Analog computers exist commercially believe it or not, but for niche applications and essentially as coprocessors.

Re: Advances in semiconductors are feeding the AI boom

#25

We already have a 4 trillion transistor "GPU" in the Cerebras WSE-3 (wafer-scale engine), used in Cerebras' data centers. https://www.youtube.com/watch?v=f4Dly8I8lMY

how many regular sized GPU (say 4080 or nvidia L4) dies can be cut out from a full-sized wafer? I suppose thats what OP means by reaching integration density of 1T GPU.

Re: Advances in semiconductors are feeding the AI boom

#26
Wild that the human brain can squeeze in 100 trillion synapses (very roughly analogous to model parameters / transistors) in a 3lb piece of meat that draws 20 Watts. The power efficiency difference may be explainable by the much slower frequency of brain computation (200 Hz vs. 2GHz).

My impression is that the main obstacle to achieving a comparable volumetric density is that we haven't cracked 3d stacking of integrated circuits yet. Very exciting to see TSMC making inroads here:

> Recent advances have shown HBM test structures with 12 layers of chips stacked using hybrid bonding, a copper-to-copper connection with a higher density than solder bumps can provide. Bonded at low temperature on top of a larger base logic chip, this memory system has a total thickness of just 600 µm...We’ll need to link all these chiplets together in a 3D stack, but fortunately, industry has been able to rapidly scale down the pitch of vertical interconnects, increasing the density of connections. And there is plenty of room for more. We see no reason why the interconnect density can’t grow by an order of magnitude, and even beyond.

It's hard to imagine not getting unbelievable results when in 10-30 years we have GPUs with a comparable number of transistors to brain synapses that support computation speed 10,000x faster than the brain. What a thing to witness!

Re: Advances in semiconductors are feeding the AI boom

#27
post #5
post #2

Cerebras WSE-3 contains 4 trillion transistors and 8 exaflops per sec, 20 PB bandwidth. 62 times the cores of an H100.. 900,000. I wonder if the WSE-3 can compete on price / performance though. Interesting times!

Is anyone actually using those WSEs in anger yet? They're on their third generation now, but as far as I can tell the discussion of each generation consists of "Cerebras announces new giant chip" and then radio silence until they announce the next giant chip.

Problem is Software. You can put out a XYZ trillion monster chip that beats anything hardware wise, but it is going nowhere if you don't have the tooling and massive community (like Nvidia has) to actually do some real A.I. stuff.

Re: Advances in semiconductors are feeding the AI boom

#28

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

Analogue circuits can be a real pain in the butt, imagine if integer overflows destroyed the whole ALU, haha. Transistors are already much smaller than neurons. And of course the brain doesn’t have a clock. And neurons have more complex behavior than single transistors… The whole system is just very different. So, this doesn’t seem like a strategy to get past a boundary, it is more like a suggestion that we give up o…

[dead]

Re: Advances in semiconductors are feeding the AI boom

#29

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

> I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. I have some theories that this isn't necessary. 1.) Just because the brain is a general-purpose machine great at doing lots of things, doesn't mean it's great at each of those things. Like when two people are playing catch, and one of them sees the first fragments of a parabola and estimates where the ball is going to land-…

I'm also in the camp that believes that we won't reach agi without significantly more compute. I think that consciousness is an emergent property, just like i think life itself is an emergent property, Both need a certain set elements/systems to work. The secret sauce may look simple when we recreate the right conditions, but it's not going to be something that makes it possible without the right hardware so to speak.
Post reply on HN