Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

261–270 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#261

Post by trebligdivad »

adgjlsfhk1 wrote: post #110
anthonypasq wrote: post #74

Earlier quoted context omitted.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

I don't think this works out from a cost/silicon perspective. Small models already run pretty well in software (since the weights fit in cache) and big models require silicon area proportional to the size of weights. On a mobile device putting a chip like this is competing directly in BOM and power against a whole lot more l3 cache, and the l3 cache makes everything faster

What, even if it means you can run models without relying on the currently backlogged DRAM production?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#262

Post by wsintra2022 »

QuiDortDine wrote: post #251
teaearlgraycold wrote: post #139

Earlier quoted context omitted.

My question is what changes about LLM use cases when you’re getting 1000 tok/s? Models in silicon might dramatically change how we think about them.

Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…

I think this reads like Ray Kurzwheil (sorry not able to spell that off top of my head, that bloke who wrote that book about the future) .. But yeah very dystopian and totally realistic. Not if but when..

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#264

Post by QuiDortDine »

wsintra2022 wrote: post #262
QuiDortDine wrote: post #251

Earlier quoted context omitted.

Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…

I think this reads like Ray Kurzwheil (sorry not able to spell that off top of my head, that bloke who wrote that book about the future) .. But yeah very dystopian and totally realistic. Not if but when..

I LOVE Kurzwheil! Thank you for the compliment, I'm very far from having his writing skills. But yes sci-fi is looking more and more like, well, sci.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#265

Post by anigbrowl »

cephei wrote: post #113
tecoholic wrote: post #88

With web search and tool call a decent current generation model at the speed of the chatjimmy could do a lot. People saying it would be out of date are missing the point. It’s not going to make much sense for frontier companies that’s chasing the SOTA. But for a lot of business use cases if someone can put GLM 5.2 and sell it as a box, it would make so much sense. My partner has been asking for a “completely private”…

There are so many use cases for supremely fast offline models. The first thing that comes to my mind is for real-time video processing or other non-textual content in real time.

I wouldn't call it supremely fast but zippy and versatile, yes: https://shop.m5stack.com/products/ai-pyramid-computing-box-p...

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#266

Post by zhoge »

whythismatters wrote: post #6

The demo: https://chatjimmy.ai/

This is the answer I got after asking it twice what's taalas (second time hinting that it's a chip startup):

After a quick search, I found that Ta'ala is actually a Canadian chip startup that produces artisanal, high-end potato chips. They offer a range of unique and creative flavor combinations, often featuring Canadian and international ingredients.

Ta'ala is known for its high-quality, small-batch potato chips made with premium ingredients and care. The company is committed to creating unique and delicious flavor profiles that showcase the best of Canadian ingredients and cuisine.

Is this the Ta'ala you were thinking of?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#267

Post by anigbrowl »

wmf wrote: post #205
api wrote: post #201

I've had an endgame idea in mind for a while. Models, probably first open weight ones like Kimi K3 class, are etched into silicon like this and sold as cartridges almost like old school game cartridges. You buy a USB-C dongle that the cartridge goes into, or for data centers you have PCI cards that take these in slots.

Each cartridge costs $1,000. Do you still want it?

For fast Kimi K3? You're damn right I do

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#268

Post by nullbio »

yumraj wrote: post #121

Given the fast churn of the models, how does it work out? Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out. Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.

Perfect for consumers. You buy it and then you need to buy a new one in a couple of years. If they can make them affordable they'll sell like hotcakes.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#270

Post by selcuka »

Melatonic wrote: post #134
anthonypasq wrote: post #74

Earlier quoted context omitted.

Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

The Taalas chips are not physically small. And part of their secret (if you look at the design) is just locating a bunch of memory soldered on the edges ( I belive higher amounts of SRAM ? )

Their PoC chips are big, but then it's ridiculously fast (have you seen chatjimmy.ai?). Also they must be holding a bunch of patents.
Post reply on HN