AMD acquires Taalas to boost inference performance by etching models in silicon • Hacker News Board

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#731

Post by nomel » Mon, Aug 10, 2026, 9:05 PM UTC

twobitshifter wrote: post #708

nomel wrote: post #191

Earlier quoted context omitted.

I don't follow. How is that related? GPUs don't have fixed memory. You don't throw them away when you want to load a new model. NVIDIA will probably give us a new GPU when someone competent in the free market decides they want wheelbarrows full of money. Unfortunately, AMD is entirely, incomprehensibly, incompetent, to the point where I can only assume they're colluding with Nvidia, behind the scenes.

Point being that hardware generations can be very quick and as updates get made GPUs go out of date and can’t run the latest models. All types of hardware consumer of otherwise are always improving, which means that tying a model the hardware is not going to lead to increased obsolescence, any more than the hardware itself does. If an AI is general purpose then there is no problem with only having that one model bake…

I think there's some confusion about what's going on here.

A GPU is general purpose, for inference sake. You can run any model that can fit in it. It will be obsolete, as all hardware eventually is, but a 3090 today is more useful than a 3090 two years ago, because small models have improved significantly.

Hardware as a model can run exactly one model, ever. You can't try a fine tune, and can't try the new similarly sized model that's better than all then others you've ever tried. You can run exactly one set of weights, with the architecture it shipped with, because everything is fixed.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#732

Post by topspin » Tue, Aug 11, 2026, 12:29 PM UTC

mdp2021 wrote: post #725

topspin wrote: post #723

Earlier quoted context omitted.

> That can only work when there is physical capacity for improvement though. There are great opportunities for advancement. Both in the physical hardware and in how and where it's deployed and powered. Consider this, as only one point: there hasn't really been a demand for advancements in ROM. RAM has been scaling at approximately Moore's law rate, and nonvolatile R/W storage has been sedately scaling, but there hasn…

Sure, but actually, our current need is not really for "ROM": it is for "CiM", compute-in-memory - we want to minimize the data movement bottlenecks. That some implementations could be read-only is actually a disadvantage. Clearly there are possibilities, some of them proven (proof-of-concept, in-production etc.) - but taking for granted "Moore's law" like spaces for them may not be founded on what we know at this st…

> our current need is not really for "ROM"

A fast, low power ROM is the key ingredient to near term local inference with large models at low power. If I could offer you a $500 ROM that provided the model data for frontier inference on power similar to a desktop GPU, you would buy it, and consider it a bargain, even when it came time to pay another $500 for the upgrade.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#733

Post by Closi » Tue, Aug 11, 2026, 2:24 PM UTC

paulryanrogers wrote: post #713

fennecbutt wrote: post #697

Earlier quoted context omitted.

>Yet the answers will get outdated quickly whilst the silicon is fixed. Bro is living in 2020 before rag was widely introduced.

I was told updates require replacing at least two layers of metal, though not whole thing. Was that not accurate? Can you say more?

You can change the prompt, and give the agent tools, and that keeps the agent up to date.

The agent doesn't know the date, or know what hotels there are in Montreal, it sees:

> System: You are an AI agent. The date is 11th August 2026. Your knowledge cut-off is March 2023. User is based in . If you need to search for something to support the user say {search:} and a list of options will be provided along with instructions on how to access. Or say {help} for a full list of commands.

> User: Can you help me find hotels in Montreal for next weekend?

The AI then interacts with the tools given in the base prompt, which can obviously be updated. So it then goes:

> AI: Of course, let me search for that. {search: hotels in montreal for 16th August}

> System: [Provides list of websites]. Say {read[n]} to read option or say {start subagent:) to register subagent.

> AI: {start subagent: List hotels on booking.com available on 16th August}.

[etc etc, then eventually]

> AI: Yes I have searched for you and I found a few options!

While you can't change embedded knowledge, a good model knowing that the date is 5th January 2040 can infer certain things (e.g. while it might not have been trained on certain deaths, it can probably guess that it should search before answering if it means a person would be 102 and their last information is from 2024)

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#734

Post by Closi » Tue, Aug 11, 2026, 2:55 PM UTC

sensanaty wrote: post #712

Closi wrote: post #561

Earlier quoted context omitted.

Scaling is definetly hard - but there is no absolute requirement to put huge flagship models into this technology (although it might be possible over time). A fairly dumb but FAST model has it's own totally distinct use-cases even if it can't be scaled in size. Think about a LLM-infused-Alexa where the response time is instant. Where you can request it looks at hotel options in Montreal, and it starts answering in ha…

And outside of idiotic demos, who exactly is going to ask an LLM to look at hotels in Montreal for them? This usecase has never made sense to me in the slightest

I mean it was just an example, but I get AI to do things like this all the time... ChatGPT planned my latest work trip by looking through flights, hotels and pulled together 3 options of itineraries for me, and wrote an email I could send to my client with estimated costs etc.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#735

Post by mdp2021 » Tue, Aug 11, 2026, 3:43 PM UTC

topspin wrote: post #732

mdp2021 wrote: post #725

Earlier quoted context omitted.

Sure, but actually, our current need is not really for "ROM": it is for "CiM", compute-in-memory - we want to minimize the data movement bottlenecks. That some implementations could be read-only is actually a disadvantage. Clearly there are possibilities, some of them proven (proof-of-concept, in-production etc.) - but taking for granted "Moore's law" like spaces for them may not be founded on what we know at this st…

> our current need is not really for "ROM" A fast, low power ROM is the key ingredient to near term local inference with large models at low power. If I could offer you a $500 ROM that provided the model data for frontier inference on power similar to a desktop GPU, you would buy it, and consider it a bargain, even when it came time to pay another $500 for the upgrade.

> A fast, low power ROM is the key ingredient

Surely it is clear to you that Read-Only /Memory/ does not /compute/, and our need is to compute through the data in the memory... That is CiM - a technology not that similar to ROM... Because a plain ROM does not solve problems in this area...

In other words,

> If I could offer you a $500 ROM that provided the model data

Then I would have a physical token containing what I already had as a file, and the problem of running that file into something efficient would remain... Because the ROM does not "run" its contents...

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#736

Post by yencabulator » Tue, Aug 11, 2026, 5:13 PM UTC

techpression wrote: post #417

kzrdude wrote: post #390

Earlier quoted context omitted.

The investment in bigger machines at the fab might set you back billions. I don't know about the lithography technology either, how easy you can scale it to larger wafers?

You also need to worry about yields, Apple, AMD etc can sell ”bad” chips as lower core versions, if you’re depending on whole wafer you have little room for error.

Cerebras does the exact same thing, disabling the bad tiny cores on their wafer-scale and routing around them in their compute network. The question is how much extra capacity do they have to plan for, and does the cost of that still make sense, with the yield they can get.

Cerebras pretty much has to be at the end of what can fit on a single wafer. Larger wafers would require retooling one of the most up-front-expensive industries, and denser is not arriving fast enough. I expect their scale-out story to rhyme with NVidia et al working at rack scale and beyond, just denser. A rack of Cerebras has 400 G networking for two wafers today.