On thing with this enormous speed is that you could create multiagent validation and correction and still have it super fast and achieve greater quality thanks to speed alone.
AMD acquires Taalas to boost inference performance by etching models in silicon
491–500 of 712 posts
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#492I'm surprised there's not more discussion about potential inflection points here. When technology gets faster, it opens up whole new classes of UX that were hard to predict For example, faster internet didn't mean being able to view 100x as many HTML4 web pages. It brought SaaS, streaming media and interactivity. I'm not good at predicting, but some ideas: 1. All information gets augmented in real time with personali…
Yes, this is a great point and it’s even one that the Cerebras CEO spruiks for his own chips. Talaas has gone furthest on the spectrum here, so seems to have the most potential to evolve the use cases.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#493Earlier quoted context omitted.
I think this would make sense for consumer hardware, not for AI companies. AI companies constantly update/change stuff, new models come out, new requirements, etc. But if you ship an "ai-powered" dishwasher, it can come with the chip built-in to do computer vision and precisely target each spot, and will be sold as-is with no updates.
It makes fairly little sense for consumer hardware, unless you're embedding an absolutely tiny model. Since you're embedding the entire model in the chip itself, required chip size scales directly with model size. The reason they have llama3.1-8B as their model in https://chatjimmy.ai/ (their demo) is because it's very hard to do anything bigger than that, you literally start running out of physical space on the chip…
And yield. Good luck getting good yield out of such ginormous chips.
That said, I've been wondering if they could go with multiple smaller ones instead. Like one per layer maybe even?
What are your thoughts on that? You seem to be more qualified than me on that matter.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#494Earlier quoted context omitted.
It gave the correct answers to both questions for me: > Bruce Lee was born in San Francisco, California, USA on November 27, 1940. > Bruce Lee's father was a Chinese opera singer That being said, this is not a good test. It is a language model (a very small one), not an encyclopedia. ChatJimmy interface is just a tech demo. Without tool calling functionality we can't expect it to be factually correct.
if it's baked into silicon how can you two get different answers?
It still works the same way other LLMs do, by outputting the probability distribution over the possible completions (The weather is ... (sunny (50%), cloudy (50%))). Then the next token is sampled from this probability distribution (in our example the next word could be "sunny" or "cloudy" equally likely), which can result in different outputs every run.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#495Earlier quoted context omitted.
It barely works even today, like Siri is laughably bad.
I mean the examples he gave definitely work. Mostly well I'd say as they are pretty primitive. What Siri is missing is more logical solutions and answers for recipes, etc (still suck even with chatgpt integration).
Works, sometimes. But they can fail spectacularly and unexpectedly even on very basic questions/instructions, like so simple that a hand-coded word-matching style logic could get them right 20 years ago.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#496Earlier quoted context omitted.
That's actually a really good point... There's currently zero incentive to buying more hardware, and that's one very good reason do have a new one.
But this is already happening with iPhones. Apple is touting on-device AI and only the latest phones offer the full capabilities. Newer phones will be able to run better models, so the incentive is there as soon as someone makes the killer app that only makes sense when the model is running locally on your phone.
> as soon as someone makes the killer app that only makes sense when the model is running locally on your phone.
I expect this to be around the time when we're finally ready to travel to Mars.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#497Earlier quoted context omitted.
1. How come you didn't make your implementation public? You could be a millionaire now. 2. Especially if AMD has the technology to do what Taalas does, it makes a ton of sense for AMD to acquire Taalas: remove them from the market. Make sure nobody else (Intel, Huawei, Alibaba, NVIDIA, etc) acquires them. It could have been a great acquisition for a rebirth of BlackBerry btw.
It will be, I'm just not happy with my blog post before making it live. The blog will have a live interactive chat and a link to the repo with the HDL. I don't think anything I did was particularly novel, as I really just wanted to see how fast I could push a commodity FPGA to it's limit. Scaling to an ASIC or getting into the billions of params is where the real engineering is! This was just a side project for a sid…
What FPGA are you using? Why?
What exactly did you implement? A full LLM? A subset of it, which collaborates with something running on CPU or GPU? Which LLM? Why?
What language did you use to implement your thing: VHDL, Verilog, Vitis, something else? Why?
I can think of at least 10 blog posts that I'd write before I write a single line of code. Publish early, publish soon ;-)
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#498Earlier quoted context omitted.
Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…
I tried it. I asked where Bruce Lee was born. It stated he was born in Hong Kong. I challenged it and it went further naming a hospital there. I stated he was born in San Francisco and it apologized and then said his father was a missionary traveling in America, which was also wrong. Bruce’s father was a famous Cantonese Opera singer and actor. This model had zero information right, while being fast in responding. Un…
Using LLMs for information retrieval is the most stupid thing one can do. Especially when old methods work much better.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#499The demo: https://chatjimmy.ai/
I was thinking the other day if we could use something like this "old" 8B model, and run 20 or 30 calls at the same time (or in sequence, we wont notice) and use and use the best result. Basically tiny agents that do tiny things but VERY fast.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#500The demo: https://chatjimmy.ai/
This is the only demo of 2026 which has blown my mind.
If we can get to this speed with reasoning models, man... I can't even imagine the impact.