Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

371–380 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#371

Post by wyrdcurt »

stingraycharles wrote: post #304
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Didn’t Anthropic acquire Cerebras? Seems like a move into the same direction. I also think that etching models into ASICs may be a bit too inflexible for what OpenAI and Anthropic want.

No, that's backwards. OpenAI are the ones investing in Cerebras. Part of the deal is that they can't sell to Anthropic.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#372

Post by bluezly »

syntaxing wrote: post #349
wmf wrote: post #146

Earlier quoted context omitted.

somewhere = an AWS data center with multiple layers of security and NDAs They won't sell/rent/license the weights to an end user at any price because they don't trust your security.

I work in embedded space. Just because it’s in hardware doesn’t mean you can’t “protect” it. Most modern software (regardless if it’s hardware or not) can be cryptophically signed.

Signing protects authenticity and integrity, but it doesn’t really solve confidentiality.

If the weights are physically encoded in hardware and the attacker owns the device, the problem becomes hardware extraction: decapping, probing, imaging, side channels, etc.

You can make that very expensive, but it’s still a very different security model from keeping the weights in a datacenter.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#373

Post by ReactiveJelly »

chorizo wrote: post #350
klodolph wrote: post #348

Earlier quoted context omitted.

I thought DRAM was pretty dense already. Is mask ROM that much denser?

Yes, each rom bit can be a transistor or even a diode with a decoder circuit. Simplest Dram cell is capacitor+transistor - and you need a clock, refresh circuit etc. Someday, I imagine model weights could even be encoded as analog resistors (memristors or similar) for even greater density

Hm. I wonder how many relays I'd need to make a physical MNIST classifier. That'd be dope

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#374

Post by __MatrixMan__ »

linzhangrun wrote: post #269

Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.

Last month: agents spend 4 days on a hack, humans spend 3 weeks (so far) digging through the slop to figure out what happened

Next time, one of those number will be smaller, and the other will likely be bigger. How long before the analysis side gets too overwhelming to bother with? Probably less than 6 years.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#375

Post by LarsDu88 »

teaearlgraycold wrote: post #139
adgjlsfhk1 wrote: post #110

Earlier quoted context omitted.

I don't think this works out from a cost/silicon perspective. Small models already run pretty well in software (since the weights fit in cache) and big models require silicon area proportional to the size of weights. On a mobile device putting a chip like this is competing directly in BOM and power against a whole lot more l3 cache, and the l3 cache makes everything faster

My question is what changes about LLM use cases when you’re getting 1000 tok/s? Models in silicon might dramatically change how we think about them.

Massive economic simulations with thousands if not millions of agents to front run the global economy and stock market.

Fully interactive realtime NPCs in videogames at scale.

Recommender systems that simulate individual consumers.

Crazy shit

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#376

Post by LarsDu88 »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

It depends on how quickly you can bake new architectures.

Text diffusion might be a disruptor here, but let me just say the most cutting edhe form of image diffusion (JiT and DiT) right now is just a big fat stack of alternating attention and MLP matmulls. Not theoretically hard to bake

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#377

Post by hahahaa »

dyzone wrote: post #366
linzhangrun wrote: post #269

Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.

It tells me that they have some kind of insider knowledge that the models have hit their limits and won't be getting much better, and it makes sense economically speaking to just bake the current models and use them for the next 5-10 years. Looks like we're near the top of the S curve.

A model you can run for practically no cost is a new proposition. It is the CPUification of AI. Sure there be supercomputers but you PC will be pretty super too.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#378

Post by [deleted] »

mountainriver wrote: post #369
dyzone wrote: post #366

Earlier quoted context omitted.

It tells me that they have some kind of insider knowledge that the models have hit their limits and won't be getting much better, and it makes sense economically speaking to just bake the current models and use them for the next 5-10 years. Looks like we're near the top of the S curve.

What would possibly tell you that?

[deleted]

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#380

Post by hahahaa »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

They still make 6502s right.
Post reply on HN