61–70 of 137 posts
On a slightly different tangent, is anyone working on analog machine learning ASICs? Sub-threshold CMOS or something? I mean even at the research level? Using a handful of transistor for an analog multiplier. And get all of the crazy fascinating translinear stuff of Barrie Gilbert fame. https://www.electronicdesign.com/technologies/analog/article... https://www.analog.com/en/resources/analog-dialogue/articles... http…
Earlier quoted context omitted.
> The CUDA moat is real. I don't know. The transformer architecture uses only a limited number of primitives. Once you have ported those to your new architecture, you're good to go. Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA.
It is beyond porting, it is mentality of developers. Change is expensive and I'm not just talking about $ value. > Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA. That's exactly what I'm saying. __they__ is the keyword.
Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.
The first revisions were stuff made by qualcomm right? I don't think we have much data on how much customizations they make and where they their IP from, but given how much of the Tensor cores comes from Samsung I think it's safe to say to assume that there is a decent amount coming from some of the big vendors.
Not sure about the mobile SoCs
Made where? Isn’t foundry capacity the limiting factor on chips for AI right now?
On a slightly different tangent, is anyone working on analog machine learning ASICs? Sub-threshold CMOS or something? I mean even at the research level? Using a handful of transistor for an analog multiplier. And get all of the crazy fascinating translinear stuff of Barrie Gilbert fame. https://www.electronicdesign.com/technologies/analog/article... https://www.analog.com/en/resources/analog-dialogue/articles... http…
analog neural network hardware
physical neural network hardware
Put "this paper" after each one to get academic research. Try it with and without that phrase. Also, add "survey" to the next iteration.
The papers that pop up will have the internal jargon the researchers use to describe their work. You can further search with it.
The "this paper," "survey," and internal jargon in various combinations are how I find most CompSci things I share.
Not suprising that the hyperscalers will make this decision for inference and maybe even a large chunk of training. I wonder if it will spur nvidia to work on an inference only accelerator.
> I wonder if it will spur nvidia to work on an inference only accelerator. Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or…
Not suprising that the hyperscalers will make this decision for inference and maybe even a large chunk of training. I wonder if it will spur nvidia to work on an inference only accelerator.
> I wonder if it will spur nvidia to work on an inference only accelerator. Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or…
And no, the NPU isn't a GPU.
Earlier quoted context omitted.
> The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and models that are optimized for older hardware can provide similar gains to models that are run on unoptimized but more performant hardware. Is anyone else getting crypto flashbacks?
The difference is crypto wasn't memory and throughput dependent. That's why a small asic on a USB stick could outperform a GPU.
The teams that work on custom ASIC design at (eg.) Broadcom for Microsoft are basically designing custom GPUs for MS, but these will only meet the requirements that Microsoft lays out, and Microsoft would have full insight and visibility into the entire architecture.