Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

151–160 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#151

Post by wmf »

cubefox wrote: post #123

> At 20 billion parameters per chip, you’d need just 50 accelerators to support a trillion-parameter model I don't see any evidence that this is possible. From my understanding, the whole model needs to be on a single chip. Which rules out any popular frontier models with several trillions of parameters. Even smaller sub-frontier models have hundreds of millions of parameters, so these would be ruled out as well.

The methods for splitting weights across multiple chips are well established. Groq/Cerebras can't hold a model on one chip either.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#153

Post by all2 »

kevin_thibedeau wrote: post #40
syntaxing wrote: post #11

Honestly, this is starting to make more and more sense. SOTA models are starting to converge to certain architecture and capabilities. I wouldn’t be surprised we end up with a base model ASIC + “fine tune” card where it’s a physical LoRA style adapter.

Then we can have machine psychologists pull cards when they run amok.

You have a robot. You need it to be smarter. You buy a new model cartridge (probably a PCIE 9.x). Now you need some domain specific skills. You'd like it to be able to cook, and you'd like it to not dent your walls anymore. You buy 'improved spatial reasoning LORA' card and 'Gordon Ramsey's Chef ULTRA9000' card.

Now your robot can respond sarcastically when you ask for chicken nuggets. Again. It also doesn't dent your walls anymore.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#154

Post by hamdingers »

karmasimida wrote: post #85
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

A model can't be updated, and a chip that is only relevant for 6 months at max?

One of these chips smart enough to take orders at a drive-thru would be relevant for a decade, minimum.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#155

Post by alightsoul »

wmf wrote: post #133
alightsoul wrote: post #104

Earlier quoted context omitted.

Because Openai and anthropic are not hardware companies. They outsource that to Broadcom and AWS' Annapurna labs.

OpenAI and Anthropic are both designing ASICs.

So they have decided that putting a small LLM on a phone would backfire because people would have a negative perception of their cloud models. Pretty sure AMD will use these taalas chips in data centers, not phones

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#156

Post by zxspectrum1982 »

mikeayles wrote: post #21

AMD could have saved their money and used their own hardware! I've got a language model doing 60k tok/s on AMD hardware already, a Xilinx Kria K26 SOM, with the weights baked into URAM/BRAM with zero DRAM in the token loop. Same thesis as Taalas: single-stream decode is bandwidth bound, so stop fetching weights from far away. Caveats stacked high, obviously. It's 3.16M parameters (tinystories, and I also have a kevin…

1. How come you didn't make your implementation public? You could be a millionaire now. 2. Especially if AMD has the technology to do what Taalas does, it makes a ton of sense for AMD to acquire Taalas: remove them from the market. Make sure nobody else (Intel, Huawei, Alibaba, NVIDIA, etc) acquires them. It could have been a great acquisition for a rebirth of BlackBerry btw.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#157

Post by prinny_ »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

They expect a sort of breakpoint at which each subsequent model version will only be marginally better than the previous ones, thus allowing them to retain their value for some time. Their business doesn’t work if each year the new model demolishes the previous one in terms of performance.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#159

Post by daishi55 »

msteffen wrote: post #122

This is neat but IMO a little crazy. Something I personally haven’t seen much of, in all the discussions of model benchmarks and AI breakthroughs, is a distinction between “peak performance” and “reliable performance”. The “peak performance” of frontier models is very high: they’re solving open math problems, analyzing large codebases, etc. But my subjective impression is that “reliable performance” is mid at best: o…

> out of 100 random questions I might think to ask, it’s likely to say something wrong or stupid a handful of times at least.

What are some examples?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#160

Post by mdp2021 »

CircuitSeuss wrote: post #130
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Apparently Anthropic is moving that way: https://arstechnica.com/ai/2026/08/anthropic-confirms-plans-...

Not necessarily: it is relevant to Taalas only if it is a compute-in-memory architecture.

The Jalapeño mentioned («Anthropic is not alone in walking this path») in the article is still a classical Von Neumann architecture.

And Taalas' idea makes sense in a perspective of scale - producing a large number of cards; "for internal use" (a lower order of items) means a high production cost.

Post reply on HN