AMD acquires Taalas to boost inference performance by etching models in silicon • Hacker News Board

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#691

Post by fulafel » Sat, Aug 08, 2026, 5:40 AM UTC

briansm wrote: post #690

phonon wrote: post #590

Earlier quoted context omitted.

A full wafer like Cerebras is about 60x that, and N2P has about 3x the transistor density. So right now it's technically feasible to etch a 1.4 trillion parameter model. So roughly DeepSeek-V4-Pro class. Imagine that running a factory, for example.

Cerebras have special techniques to work around etching errors / bad cores on their wafers. This is possible since their wafers are effectively hundreds of identical copies of redundant cores. Can't do that for a globally unique model. Etching failure in that situation would be like brain-damage in a human, all sorts of weird effects would start appearing.

There's several ways to engineer around that as the errors are detectable. There's a big literature on how to trade off speed or transistors for error correction. [1]

(Is Cerebras doing something novel? CPUs and memory blocks have been doing those things for a long time too, since the error rate is otherwise too high for normal size chips as well)

[1] see eg https://www.vlsimentor.com/dft/redundancy-bisr to get some basic concepts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#692

Post by nomel » Sat, Aug 08, 2026, 6:32 AM UTC

try-working wrote: post #661

nomel wrote: post #637

Earlier quoted context omitted.

I do. The point is inference speed and power, making previously impossible local inference possible. A side effect of that hardware optimization is fixed capabilities. You've confused engineering compromise for malice, and reversed the purpose. For the model capabilities and inference power draw, what alternative do you see to a (at least mostly) fixed hardware model?

What I'm saying is that Apple will use these type of models etched into chips, and they will do it because it drives obsolescence, so they can shorten the upgrade cycle. They will do it because they figure out it's good for them.

You've confused engineering compromise for malice, and reversed the purpose. For the model capabilities and inference power draw, what alternative do you see to a (at least mostly) fixed hardware model?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#693

Post by B-Con » Sat, Aug 08, 2026, 7:03 AM UTC

daishi55 wrote: post #284

wmf wrote: post #202

Earlier quoted context omitted.

There's a benchmark for this and a lot of models get negative scores because they're so unreliable: https://artificialanalysis.ai/evaluations/omniscience

I wanted some examples they actually experienced. Because I use these things daily and haven’t seen a hallucination in a long long time.

Not OP but I've tried using all the major frontier models to find niche items in a field that I collect. The models get confused and hallucinate items that sounds complete reasonable, but don't actually exist. Usually it's an amalgamation of several real products.

My prompt is akin to "recommend with ". The first 3-ish results are about right, and then 7 of the next 10 are hallucinations and the LLM clearly can't throw up its hands and say "I got nothing".

I'm sure this is a hard problem because of a) how many items there are, b) how much overlap there is between product names, descriptions, manufacturers, different versions of the same product, etc, so keeping them distinct in the model's memory is probably hard, and even worse if it is dynamically fetching and summarizing content then it will be very easy to conflate different items, and c) LLMs are known for not working well on the edge cases with few examples.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#694

Post by senderista » Sat, Aug 08, 2026, 7:09 AM UTC

jodrellblank wrote: post #340

senderista wrote: post #50

Earlier quoted context omitted.

Wow, feels like Google web search in 1999.

or LiveGrep fast search of the Linux kernel source code with regex support: https://livegrep.com/search/linux

Wow, I want something like that for my company's codebase.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#695

Post by LarsDu88 » Sat, Aug 08, 2026, 7:30 AM UTC

manmal wrote: post #394

LarsDu88 wrote: post #375

Earlier quoted context omitted.

Massive economic simulations with thousands if not millions of agents to front run the global economy and stock market. Fully interactive realtime NPCs in videogames at scale. Recommender systems that simulate individual consumers. Crazy shit

About your first example, isn’t the butterfly effect preventing this from being useful? One agent in your simulation decides to sell, and starts an avalanche, that won’t happen in reality?

Run the sim many times... faster than it can run on actual humans and compute a probability density for specific events.

Better yet use it to dimulate counterfactual phenomena like market manipulations ypu intend to enact...

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#696

Post by rnewme » Sat, Aug 08, 2026, 7:58 AM UTC

vel0city wrote: post #644

rnewme wrote: post #633

Earlier quoted context omitted.

You load it, tell it what's in and it sets the program, tells you what it set and why. You approve and off it goes

I do that today by turning a dial and pressing the start button.

Not saying it's the superior way, but to be honest I wouldn't know how to start some washing machines without consulting manual first, having very dumb simple appliance myself.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#697

Post by fennecbutt » Sat, Aug 08, 2026, 8:16 AM UTC

paulryanrogers wrote: post #678

Closi wrote: post #561

Earlier quoted context omitted.

Scaling is definetly hard - but there is no absolute requirement to put huge flagship models into this technology (although it might be possible over time). A fairly dumb but FAST model has it's own totally distinct use-cases even if it can't be scaled in size. Think about a LLM-infused-Alexa where the response time is instant. Where you can request it looks at hotel options in Montreal, and it starts answering in ha…

> Where you can request it looks at hotel options in Montreal, and it starts answering in half a second Yet the answers will get outdated quickly whilst the silicon is fixed.

>Yet the answers will get outdated quickly whilst the silicon is fixed.

Bro is living in 2020 before rag was widely introduced.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#698

Post by mdp2021 » Sat, Aug 08, 2026, 8:19 AM UTC

sgc wrote: post #685

mdp2021 wrote: post #239

Earlier quoted context omitted.

> What does it take to go from here to a model on a pcie card or an m.2 card It is already that. > Will "intelligence" become much like a gpu As an option among the implementations. > Are we a couple years away They could mass produce now, but it makes no sense at this rate of improvements in the models.

Thanks for answering. This is an 8b model, which are mainly curiosities outside niche tasks. I guess I am asking how far we are away from having today's more generally useful frontier model equivalents widely available for everyday users in their personal pcs/laptops via a single pcie or m.2 drop in.

> how far we are away from having

It depends on AMD now. What was planned after the 8b was a ~30b, which is already sufficient (or more, when running at ultra-high speed).

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#699

Post by girishso » Sat, Aug 08, 2026, 9:23 AM UTC

paulryanrogers wrote: post #680

dpedu wrote: post #586

Earlier quoted context omitted.

350x is only about 10-20 years of improvement, using CPU FLOPS as the benchmark.

Haven't CPUs largely plateaued? They're just getting bigger, more power hungry, and multiplying cores. Physics has hard limits and Moore's law is long dead.

AFAIK these don't have to be CPUs. ASICS are well suited for this purpose. This will be the most cost effective way to run LLMs.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#700

Post by surrealist » Sat, Aug 08, 2026, 10:31 AM UTC

mafuy wrote: post #655

dabbz wrote: post #636

I see a lot of discourse about it being fast-to-deprecation. But I see it a different way personally. Modern LLMs are trying to do more with less. Focus on doing the right thing the first time. Even if we squeeze dumb LLMs, the significantly faster speed means quicker iterations. So a bad decision doesn't cost the time and inference costs that it cost before. It theoretically changes the scale of errant token spend.…

Sounds to me like you would hire 20 barely-paid interns instead of 2 competent programmers.

I would hire malleable interns fit for replication, and teach them all the things they would need to know. But for some reason I was forbidden to do that, so I'm forced to settle for cheap LLMs instead.