Earlier quoted context omitted.
Wow, feels like Google web search in 1999.
If you still want the experience, go and browse McMaster Carr. Wizards designed that website.
Oh I have, though not for a while.
AMD acquires Taalas to boost inference performance by etching models in silicon
321–330 of 712 posts
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#322Earlier quoted context omitted.
Just to see how fast it is try chatjimmy.ai
It is really fast and ... really hallucinates. I asked "Does the Wang corporation still exist? If not, what happened to it?" and it replied (in part): "Yes, the Wang Corporation, the company that originally developed and marketed the Wang 2200 computer, still exists as a rebranded company under the name PPL (Precision Pencil and Label), but it has undergone significant changes and challenges over the years. Here's a…
their tech is a mere demo to open up a new path, the day we can have some asics running a Qwen3.6 27b, this would open up new doors
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#323Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#324I have used chatjimmy before, it is incredibly fast, waiting for latest SOTA model on the chips in future. Great!
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#325What I like about this, is that it significantly increases the probability of a sci-fi scenario where you're picking up a hot chip on the black market; rumor has it, Mythos 9 weights baked in...
Plug it in, and it's a old prototype with Gemma 5 weights baked onboard. Dammit, fucked by Craigslist again!
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#326taalas is great for llama 3.x 8B models, really bad for one board serving Kimi K3, it seems like you would bottleneck at a few hundred tokens no matter what you do.... spreading the big model against multiple cards seems the only way to get into the 1k+ tok/sec range. Another thing taalas is doing is masking the model weights into the silicon itself, not a flashable firmware, which would increase latency....
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#327They moved from HBM to dedicated silicon and only got a 48x speed up? That is so, so, so much less than I would've expected. Any numbers on how it scales?
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#328Earlier quoted context omitted.
Depends on how much it costs the consumer. If I could buy a "cartridge" of Kimi K3 for 300 bucks I 100% would buy that shit asap. Even if it's "no good" after lets say 4 months still would be worth it IMO.
That's definitely super-enthousiast territory. Paying 80 bucks a month for AI is more than 99.99% of people would be willing to do
This will be considered very cheap within the year IMO. The value you get from AI is exponentially increasing and like all tech just takes some time to ramp up. Cell phones, internet and many other amenities when they came out many people were not willing to pay for but that all changed and considering how important AI tech is this will also be the case especially considering if its 100% private such as for that cartridge.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#329Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#330Earlier quoted context omitted.
Considering the rate of model development and rail hopping, seems like baking models into silicon is speed-running obsolescence.
I'd gladly pay for a Claude Opus 4.6 Thinking High in silicon and use it for 1-2 years. It's good enough for many coding tasks.
Not so long ago, I was good enough for many coding tasks. But I found that things can change in a hurry.
Yes, a cheap and fast Opus4.6 can drive a lot of value in current context. But if we continue to craft bigger-and-bigger balls of mud, Opus 4.6 may end up hitting its conceptual ceiling and unable to contribute.
Winding the clock back on your statement gives:
> I'd gladly pay for a Claude Sonnet 3.5 in silicon and use it for 1-2 years.
Man, I dunno.