Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

91–100 of 725 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#91

Post by breadislove »

syntaxing wrote: post #11

Honestly, this is starting to make more and more sense. SOTA models are starting to converge to certain architecture and capabilities. I wouldn’t be surprised we end up with a base model ASIC + “fine tune” card where it’s a physical LoRA style adapter.

we have not converged at all, if you look at how different the chinese models in terms of architecture you can guess that the labs are experimenting a lot as well. we are seeing all different types of hybrid architectures, different attention methods and so on. Of course on a high level its still a transformer but if you take a proper look we are seeing more divergence then a convergence.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#92

Post by samtheprogram »

edot wrote: post #76
2001zhaozhao wrote: post #64

Earlier quoted context omitted.

There are always uses for outdated models. Claude Code is still using haiku 4.5 from ages ago for explore subagents for instance. Not to mention production uses like customer service that only need to be "good enough"

Just looked this up, no longer true. Explore subagents inherit whatever model the parent is. And you can of course make other subagent configs.

That's solely so that you burn more money. It's totally unnecessary to assume the parent model. Sure, it could be upgraded from Haiku if there was a solid reason to, but...

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#93

Post by nine_k »

wolttam wrote: post #77
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

It's a terrible moat. You etch the silicon then nobody wants to run it in 6 months because models have advanced that much further.

Not so if it's embedded in something smart enough for its intended purpose.

Think vision, spatial reasoning, speech synthesis, even some speech analysis. Think self-driving cars (and drones) that need 10x less power for the brain, and can think at 10x situation per second.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#94

Post by mdp2021 »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

Compute the cost of producing n of them devices, imagine a fair price based on that, and see if that local, blazing fast card* can be an asset that could be replaced periodically.

*(It's local: private files managing firm oriented. It's blazing fast: it can be placed into recursive, intensive local workflows.)

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#95

Post by equinumerous »

tecoholic wrote: post #88

With web search and tool call a decent current generation model at the speed of the chatjimmy could do a lot. People saying it would be out of date are missing the point. It’s not going to make much sense for frontier companies that’s chasing the SOTA. But for a lot of business use cases if someone can put GLM 5.2 and sell it as a box, it would make so much sense. My partner has been asking for a “completely private”…

100% agree - you don't need the most up-to-date model to have something that's useful in agentic contexts. They could even produce chips with weights that make all the decision making/logical reasoning and have it delegate to other specialized agents. If it becomes cheap enough to print a run of custom chips, releasing a batch for each major advancement does not seem unreasonable for SOTA companies.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#96

Post by thepasch »

golem14 wrote: post #69
walrus01 wrote: post #45

Imagine the size of chip needed to 'etch' something like Qwen 3.6 27B in size.

Interesting thought, because it's a yield question. How tolerant are models today to a few broken weights. If tolerant, they could churn out many cheaper chips, some perhaps with slight abnormal tendencies ;)

> How tolerant are models today to a few broken weights.

Extremely! You can remove entire layers and the model will still work just fine, with barely perceptible capability losses.

I've cut/bypassed ~15% of total parameters out of Gemma 4 31B on a pod once. Still got perfectly coherent responses out of it. Certain layers are a lot more important than others, particularly early and late ones; but it's honestly astonishing how much can be cut out from the middle without destroying the model's coherence.

I didn't run any meaningful benchmarks, so I have no idea what the capability loss looks like exactly. But "produce coherent and sensible English in response to a wide variety of prompts" was definitely not among the things the model unlearned.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#97

Post by AussieWog93 »

edot wrote: post #76
2001zhaozhao wrote: post #64

Earlier quoted context omitted.

There are always uses for outdated models. Claude Code is still using haiku 4.5 from ages ago for explore subagents for instance. Not to mention production uses like customer service that only need to be "good enough"

Just looked this up, no longer true. Explore subagents inherit whatever model the parent is. And you can of course make other subagent configs.

I mean, if you could get Opus or even Sonnet 4.5 at 1000+ tok/s exploring the codebase, they would probably change that setting back.

But either way, I think GP's overall sentiment of "delegating intelligence-saturated tasks to an outdated but fast subagent" makes a lot of sense.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#98

Post by varun_ch »

VBprogrammer wrote: post #56
wxw wrote: post #31

Earlier quoted context omitted.

I freakin' love this demo. It feels magical.

I had the same reaction but then I showed it to my partner. She completely didn't get it, in her words "how can it be thinking of a good answer when it's that quick?" I tried to explain but I fear were probably going to be adding artificial sleeps to these things to convince the masses it's doing something clever.

to be fair, the model used for Chat Jimmy is not very smart, but the world where it is smart is very interesting.

It’s going to be really crazy when the bottle neck for agents is the speed of the tool calls rather than the speed of inference. Imagine an agent interacting with the terminal near instantly…

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#99

Post by xienze »

shangofox wrote: post #65
nojs wrote: post #52

Can anyone comment on the economics and likely turnaround times of this process, when it’s more mature? Would it be realistic for a frontier lab to deploy this or would the turnaround time mean the model is always too out of date? Assuming the weights and architecture are eventually stable, how much cheaper would this end up being?

I mean even if it take a few months, it'll still be out of date. But there was a hypothetical when it came up in Feb, would you want Qwen 3.5 at like 10k tokens per second. At the time people were no doubt saying yes but now 3.8 is out, is that still desirable?

There's soooo much stuff that such a model is still capable of doing in the pursuit of getting a better overall answer. Imagine a powerful research agent that blasts out dozens of the small, cheap models to fetch and summarize one page each. Then the beefy researcher model performs the final analysis.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#100

Post by flyinglizard »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

Look at it the other way: compared to the cost of training a model, the cost of making a custom ASIC is trivial.
Post reply on HN