The demo: https://chatjimmy.ai/
I freakin' love this demo. It feels magical.
AMD acquires Taalas to boost inference performance by etching models in silicon
31–40 of 710 posts
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#32Earlier quoted context omitted.
If we had deepseek v4 flash 0731 etched on a chip it would be more than capable enough and fast enough for so many people's needs, even hardcore engineer.
Will be capable and fast enough for 2-3 weeks until new sota drops
If it is capable today why would a new model change this?
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#33Earlier quoted context omitted.
It doesn’t believe it’s running on that chip, it’s arguing with me
It's running a very small, non-reasoning model at the moment. But more generally, almost all LLMs argue on the hardware/model they are/are on.
Which model? Or how many active parameters?
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#34Toronto Canada startup btw.
Works well, I remember driving by the ATI building as a kid.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#35Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#36Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#37Honestly, this is starting to make more and more sense. SOTA models are starting to converge to certain architecture and capabilities. I wouldn’t be surprised we end up with a base model ASIC + “fine tune” card where it’s a physical LoRA style adapter.
Wouldn't this mean someone with sufficient hardware could lift the SOTA model weights off the chip? Or are you saying that these chips would only be used internally by these companies and not sold to the public?
I wouldn't expect companies not sharing their weights today to be any more likely to share them if they're on hardware, this doesn't sufficiently hide weights from a local user.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#38AMD could have saved their money and used their own hardware! I've got a language model doing 60k tok/s on AMD hardware already, a Xilinx Kria K26 SOM, with the weights baked into URAM/BRAM with zero DRAM in the token loop. Same thesis as Taalas: single-stream decode is bandwidth bound, so stop fetching weights from far away. Caveats stacked high, obviously. It's 3.16M parameters (tinystories, and I also have a kevin…
Well, technically it is their hardware now...
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#39Token quantity will have a quality all its own.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#40Honestly, this is starting to make more and more sense. SOTA models are starting to converge to certain architecture and capabilities. I wouldn’t be surprised we end up with a base model ASIC + “fine tune” card where it’s a physical LoRA style adapter.
Then we can have machine psychologists pull cards when they run amok.