Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

181–190 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#181

Post by sebular »

bsaul wrote: post #82
anthonypasq wrote: post #74

Earlier quoted context omitted.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

That's actually a really good point... There's currently zero incentive to buying more hardware, and that's one very good reason do have a new one.

But this is already happening with iPhones. Apple is touting on-device AI and only the latest phones offer the full capabilities. Newer phones will be able to run better models, so the incentive is there as soon as someone makes the killer app that only makes sense when the model is running locally on your phone.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#182

Post by deadbabe »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

You could take your silicon chip and have it re-etched only with model diffs for an upgraded version.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#183

Post by SR2Z »

giancarlostoro wrote: post #127
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

ASICs is what took over Bitcoin mining, cheaper in all ways, and lasts longer than Nvidia GPUs for inference.

> cheaper in all ways,

Bitcoin mining doesn't have large memory requirements, but does have huge compute requirements. ASICs work great there because it's very straightforward to add some circuits for computing hashes. If you _also_ have to add many GB of memory, then suddenly ASICs will cost as much or more than comparable off-the-shelf hardware and they won't be faster unless you've also invested in huge memory bandwidth.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#185

Post by XCSme »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

I think this would make sense for consumer hardware, not for AI companies.

AI companies constantly update/change stuff, new models come out, new requirements, etc.

But if you ship an "ai-powered" dishwasher, it can come with the chip built-in to do computer vision and precisely target each spot, and will be sold as-is with no updates.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#187

Post by deadbabe »

sigmoid10 wrote: post #132
yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

I find speed alone would be a game changer for current models. I hardly find any task anymore that the current frontier models can't do with max reasoning after several rounds of feedback (provided sufficient instruction and the right harness). But waiting an hour or more for reasoning to finish is getting really cumbersome. If they could do the same in seconds (and for cheap of course), I'm pretty sure we'd pretty s…

Can you give some examples of these tasks that require an hour or more of reasoning?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#188

Post by dumberquestions »

msteffen wrote: post #122

This is neat but IMO a little crazy. Something I personally haven’t seen much of, in all the discussions of model benchmarks and AI breakthroughs, is a distinction between “peak performance” and “reliable performance”. The “peak performance” of frontier models is very high: they’re solving open math problems, analyzing large codebases, etc. But my subjective impression is that “reliable performance” is mid at best: o…

I think you're underestimating both their reliability for standard problems and the usefulness of that level of reliability.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#189

Post by laweijfmvo »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

pretty much everything is “1 or more versions behind” by the time it comes out. the question is whether or not it’s still useful? at some point, presumably not every application will need the latest cutting edge huge model.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#190

Post by Melatonic »

Legend2440 wrote: post #163
Melatonic wrote: post #141

Earlier quoted context omitted.

Yeah Im surprised nobody is talking about this. When everyone first saw Taalas I looked at the design and it had a big legup in physical cache availale compared to most chips. Makes you wonder how much of a benefit there is to the actual "baking" of the model vs just having a large chip with a ton of SRAM (or whatever) soldered close to the edge physically. I feel like what we really need is the ability to solder com…

What you're describing is what Cerberas does. Talaas is different, it's a true compute-in-memory architecture where the weights are stored in the connections between the transistors that perform the matrix multiply, rather than in seperate memory cells. Most of the benefit comes from this architecture; hardwiring the weights into the silicon is just the easiest way to implement it. SRAM requires too many transistors,…

Cerebras chips are massive and do have more on the edge but they dont have any top or bottom cache do they?
Post reply on HN