Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

101–110 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#101

Post by superb_dev »

anthonypasq wrote: post #74
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

From what I remember, these chips are not mobile size yet

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#102

Post by cogman10 »

nojs wrote: post #52

Can anyone comment on the economics and likely turnaround times of this process, when it’s more mature? Would it be realistic for a frontier lab to deploy this or would the turnaround time mean the model is always too out of date? Assuming the weights and architecture are eventually stable, how much cheaper would this end up being?

2 to 3 months optimistically assuming everything goes smoothly and is fully automated.

6 months or even a year if something goes wrong in the fabrication process and you need to update things.

If they do more standard asic design, it could be a lot longer as the design needs to be validated on an FPGA cluster, which would necessarily need to be very big for something like a LLM. Easily up to 2 years.

There's a reason chatjimmy isn't demonstrating newer models and why they only show of an 8B model.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#103

Post by thoughtpeddler »

walrus01 wrote: post #62
thoughtpeddler wrote: post #55

Earlier quoted context omitted.

I didn't realize there was a SchlongBench™ (but of course there is). What's it test? (asking seriously)

There isn't SchlongBench(TM) yet, it's a specific question I've been asking of differently sized models as a randomly chosen gauge of how much less commonly used knowledge is perma-baked into it. In this case a question about a specific yiddish origin slang term. Small/bad models don't know it's from middle high german or Yiddish and get its origin and meaning totally wrong (or it runs into model censorship related t…

Oh ya, this is like the approach from the Incompressible Knowledge Probes [0] paper - smart!

[0] Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity [https://arxiv.org/abs/2604.24827]

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#104

Post by alightsoul »

LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Because Openai and anthropic are not hardware companies. They outsource that to Broadcom and AWS' Annapurna labs.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#105

Post by askvictor »

karmasimida wrote: post #85
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

A model can't be updated, and a chip that is only relevant for 6 months at max?

People already buy new phones every year, this just creates even more reason to do so

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#106

Post by alightsoul »

moshun wrote: post #60
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.

Which is exactly what companies and shareholders want to increase sales.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#107

Post by [deleted] »

wolttam wrote: post #77
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

It's a terrible moat. You etch the silicon then nobody wants to run it in 6 months because models have advanced that much further.

[deleted]

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#108

Post by bradfa »

superb_dev wrote: post #101
anthonypasq wrote: post #74

Earlier quoted context omitted.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

From what I remember, these chips are not mobile size yet

A small model would be. I think that’s more the point. It’s definitely not SOTA but it’s fast and energy efficient and local.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#109

Post by thombles »

amazingamazing wrote: post #32
nurumaik wrote: post #24

Earlier quoted context omitted.

Will be capable and fast enough for 2-3 weeks until new sota drops

If it is capable today why would a new model change this?

I think it’s tongue in cheek. When I first got access to Sonnet 4.5 I remember thinking to myself “y’know if they never got any better and I just had access to this forever then that would be pretty okay”. Turns out my expectations have changed since then and I would like a higher baseline now.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#110

Post by adgjlsfhk1 »

anthonypasq wrote: post #74
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

I don't think this works out from a cost/silicon perspective. Small models already run pretty well in software (since the weights fit in cache) and big models require silicon area proportional to the size of weights. On a mobile device putting a chip like this is competing directly in BOM and power against a whole lot more l3 cache, and the l3 cache makes everything faster
Post reply on HN