Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

291–300 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#291

Post by preg_match »

mdp2021 wrote: post #136
topspin wrote: post #119

Earlier quoted context omitted.

"seems like baking models into silicon is speed-running obsolescence" Now maybe. When models are flying passenger aircraft, other prerogatives will assert themselves. When a 50TB ROM means you can impulse purchase a ChatGPT 6.3 xhigh that runs on batteries, yet more use cases will be apparent.

Well, 50TB ROM Taalas HC1 style would be apparently a 400000b transistor system through a chip sized 2.5 meters on the side... :)

Yes but have we considered employing, like, a really big block of ice? Like old-timey surgeries? What if we put a big block of ice on the 2.5 cubic meter CPU what happens then?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#294

Post by etoxin »

nullbio wrote: post #268
yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

Perfect for consumers. You buy it and then you need to buy a new one in a couple of years. If they can make them affordable they'll sell like hotcakes.

And the second hand market. I'd love to see this integrated into motherboards like RAM. Someone could have a motherboard with 4 sticks of different AI with various models. Swap, change and trade.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#297

Post by mrheosuper »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

I'm still using Opus for most daily task because Fable is too expensive.

If they begin etching Fable into silicon now and release it 2-3 years later, i can see the market for it

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#298

Post by tasty_freeze »

la6479 wrote: post #214
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Just to see how fast it is try chatjimmy.ai

It is really fast and ... really hallucinates. I asked "Does the Wang corporation still exist? If not, what happened to it?" and it replied (in part):

"Yes, the Wang Corporation, the company that originally developed and marketed the Wang 2200 computer, still exists as a rebranded company under the name PPL (Precision Pencil and Label), but it has undergone significant changes and challenges over the years.

Here's a brief overview of what happened:

    Founding and Growth: The Wang Corporation was founded by An Wang in 1969."
In fact, Wang labs was founded in 1951. PPL seems to be a made up entity. But it did generate those "facts" in 0.033 seconds. If people value speed over accuracy then I can write an LLM that is 100x faster than chatjimmy.ai and make big bucks by responding one of N canned responses to any question.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#299

Post by noisy_boy »

teraflop wrote: post #222
anigbrowl wrote: post #129

Earlier quoted context omitted.

This is only true for people who are solely focused on performance. There is absolutely a market for acceptable performance combined with predictability.

True, but predictability cuts both ways. We're all used to having to constantly update our browsers and phones to keep up with the security arms race. If a frozen model can't be updated, it will predictably remain vulnerable to any "exploits" or idiosyncratic quirks that people discover over time. Let's say, as somebody suggested in another comment, that you buy 100,000 of these chips and deploy them to run fast-food…

> Of course the real answer is to just never expose the model to situations where an adversarial input could possibly lead to an undesired output. But that drastically limits what you can do with it.

Does it though? Isn't that what CPUs are, very fast-not-so-clever computing brain surrounded by layers that protect it?

Post reply on HN