Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

111–120 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#111

Post by axus »

VBprogrammer wrote: post #56
wxw wrote: post #31

Earlier quoted context omitted.

I freakin' love this demo. It feels magical.

I had the same reaction but then I showed it to my partner. She completely didn't get it, in her words "how can it be thinking of a good answer when it's that quick?" I tried to explain but I fear were probably going to be adding artificial sleeps to these things to convince the masses it's doing something clever.

I asked it some old hardware command line questions I'd recently asked Gemini, it hallucinated parts of the answer.

The characters in the 3-act Shakespearean play had very little depth, many of the names were similar, and they were not very smart, but the simple plot was cohesive.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#112

Post by alightsoul »

2001zhaozhao wrote: post #64
nojs wrote: post #52

Can anyone comment on the economics and likely turnaround times of this process, when it’s more mature? Would it be realistic for a frontier lab to deploy this or would the turnaround time mean the model is always too out of date? Assuming the weights and architecture are eventually stable, how much cheaper would this end up being?

There are always uses for outdated models. Claude Code is still using haiku 4.5 from ages ago for explore subagents for instance. Not to mention production uses like customer service that only need to be "good enough"

Customer service has really degraded huh. 4 years ago they expected opus performance out of human call center agents

I guess losing some customers due to poor customer service is ok if the price of customer service is right.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#113

Post by cephei »

tecoholic wrote: post #88

With web search and tool call a decent current generation model at the speed of the chatjimmy could do a lot. People saying it would be out of date are missing the point. It’s not going to make much sense for frontier companies that’s chasing the SOTA. But for a lot of business use cases if someone can put GLM 5.2 and sell it as a box, it would make so much sense. My partner has been asking for a “completely private”…

There are so many use cases for supremely fast offline models. The first thing that comes to my mind is for real-time video processing or other non-textual content in real time.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#114

Post by mdp2021 »

Grosvenor wrote: post #51
encyclopedism wrote: post #22

Earlier quoted context omitted.

Imagine a multi-modal model with 1000's of tokens per second. Realtime inference for a host of applications. This is a BIG deal and will change the landscape in unfathomable ways. The https://chatjimmy.ai demo was impressive. Once models settle down this makes sense. Imagine a cartridge with a physical model on it. You purchase a cartridge and stick it in your computer/phone/server. Want to upgrade? By a new 'cartrid…

> Imagine a cartridge with a physical model on it. I can finally have my own Dixie flatline. Cool.

> Dixie Flatline

In case some did not know: also the movie (actually TV series) is finally happening.

# Neuromancer - Official Teaser ( https://news.ycombinator.com/item?id=49055037 )

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#115

Post by foltik »

snek_case wrote: post #28
VladVladikoff wrote: post #19

Earlier quoted context omitted.

Wouldn't this mean someone with sufficient hardware could lift the SOTA model weights off the chip? Or are you saying that these chips would only be used internally by these companies and not sold to the public?

The weights are very unlikely to be on the chip itself. That wouldn't work for SOTA models that are terabyte scale, even quantized. This is probably an accelerator for specific kernels in the model, but the weights are likely loaded from memory. The chip may have SRAM to store some of the weights temporarily during inference.

At least in the case of Taalas the weights are physically encoded directly on the chip.

It’s composed of 4-bit multiplier cells that compute all 16 possible results in parallel. The top metal wiring layer physically selects the one that corresponds to a multiplication with that cell’s constant weight, and routes it to the next layer.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#116

Post by tsujamin »

amelius wrote: post #68
moshun wrote: post #60

Earlier quoted context omitted.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

Not sure. You can fix the transistors but leave the connections between them open for flexibility, so you only need to change the manufacturing process for the upper masks for every new model.

Surely that added flexibility negatively impacts the density/parameter count of the model you could etch?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#118

Post by walrus01 »

AussieWog93 wrote: post #86
walrus01 wrote: post #47

Earlier quoted context omitted.

I know it's a relatively tiny model, but damn, is that thing fast. It also mostly passes the "schlong" test https://pastes.io/YcxSi8Fp

I read the paste, it got the etymology wrong, no? Schlong comes from shlang (snake), not shlemp (is this even a word? I don't speak Yiddish but couldn't find it on Google). Oxford also claim that its first recorded use was from the 60s, not the 20s; https://www.oed.com/dictionary/schlong_n?tl=true

It did get it wrong but it also got a lot farther than much more recent, but worse models like 6.7GB on disk size ternary bonsai. It at least knows it's from Yiddish. The "schlemp" appears to be a total hallucination or it's confusing it with schlep, which is not related to schlong. One of the reasons why I said it "mostly" passes the test. Something much larger on the size of qwen 3.5 122B, deepseek v4 flash or similar that runs in 120GB to 190GB of RAM in my experience will answer perfectly unless it has been ruined by something like Q2 quantization.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#119

Post by topspin »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

"seems like baking models into silicon is speed-running obsolescence"

Now maybe. When models are flying passenger aircraft, other prerogatives will assert themselves. When a 50TB ROM means you can impulse purchase a ChatGPT 6.3 xhigh that runs on batteries, yet more use cases will be apparent.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#120

Post by breuleux »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

If you’re only running models for frontier capabilities, yeah. For tasks where current models are smart enough, running them 100x faster is the most impactful improvement you can make. Consider all the things you could use a model for, but don’t, because the latency is just a bit too high.
Post reply on HN