How Taalas “prints” LLM onto a chip?
anuragk.com
How Taalas “prints” LLM onto a chip?
1–10 of 266 posts
Re: How Taalas “prints” LLM onto a chip?
#2Edit: reading the below it looks like I'm quite wrong here but I've left the comment...
The single transistor multiply is intriguing.
Id assume they are layers of FMA operating in the log domain.
But everything tells me that would be too noisy and error prone to work.
On the other hand my mind is completely biased to the digital world.
If they stay in the log domain and use a resistor network for multiplication, and the transistor is just exponentiating for the addition that seems genuinely ingenious.
Mulling it over, actually the noise probably doesn't matter. It'll average to 0.
It's essentially compute and memory baked together.
I don't know much about the area of research so can't tell if it's innovative but it does seem compelling!
Re: How Taalas “prints” LLM onto a chip?
#3This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc.
Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies. Of course, this wouldn't be useful for training models but as the models further improve, I can totally see this inside fully local + ultrafast + ultra efficient processors.
Re: How Taalas “prints” LLM onto a chip?
#4Edit: reading the below it looks like I'm quite wrong here but I've left the comment... The single transistor multiply is intriguing. Id assume they are layers of FMA operating in the log domain. But everything tells me that would be too noisy and error prone to work. On the other hand my mind is completely biased to the digital world. If they stay in the log domain and use a resistor network for multiplication, and…
The document referenced in the blog does not say anything about the single transistor multiply.
However, [1] provides the following description: "Taalas’ density is also helped by an innovation which stores a 4-bit model parameter and does multiplication on a single transistor, Bajic said (he declined to give further details but confirmed that compute is still fully digital)."
[1] https://www.eetimes.com/taalas-specializes-to-extremes-for-e...
Re: How Taalas “prints” LLM onto a chip?
#5> Kinda like a CD-ROM/Game cartridge, or a printed book, it only holds one model and cannot be rewritten.
Imagine a slot on your computer where you physically pop out and replace the chip with different models, sort of like a Nintendo DS.
Re: How Taalas “prints” LLM onto a chip?
#6> Kinda like a CD-ROM/Game cartridge, or a printed book, it only holds one model and cannot be rewritten. Imagine a slot on your computer where you physically pop out and replace the chip with different models, sort of like a Nintendo DS.
That's the kind of hardware am rooting for. Since it'll encourage Open weighs models, and would be much more private.
Infact, I was thinking, if robots of future could have such slots, where they can use different models, depending on the task they're given. Like a Hardware MoE.
Re: How Taalas “prints” LLM onto a chip?
#7Edit: reading the below it looks like I'm quite wrong here but I've left the comment... The single transistor multiply is intriguing. Id assume they are layers of FMA operating in the log domain. But everything tells me that would be too noisy and error prone to work. On the other hand my mind is completely biased to the digital world. If they stay in the log domain and use a resistor network for multiplication, and…
The document referenced in the blog does not say anything about the single transistor multiply. However, [1] provides the following description: "Taalas’ density is also helped by an innovation which stores a 4-bit model parameter and does multiplication on a single transistor, Bajic said (he declined to give further details but confirmed that compute is still fully digital)." [1] https://www.eetimes.com/taalas-speci…
That's much more informative, I think my original comment is quite off the mark then.
Re: How Taalas “prints” LLM onto a chip?
#8> Kinda like a CD-ROM/Game cartridge, or a printed book, it only holds one model and cannot be rewritten. Imagine a slot on your computer where you physically pop out and replace the chip with different models, sort of like a Nintendo DS.
A cartridge slot for models is a fun idea. Instead of one chip running any model, you get one model or maybe a family of models per chip at (I assume) much better perf/watt. Curious whether the economics work out for consumer use or if this stays in the embedded/edge space.
Re: How Taalas “prints” LLM onto a chip?
#9Note that this doesn't answer the question in the title, it merely asks it.
Re: How Taalas “prints” LLM onto a chip?
#10Note that this doesn't answer the question in the title, it merely asks it.
Yeah, I had written the blog to wrap my head around the idea of 'how would someone even be printing Weights on a chip?' 'Or how to even start to think in that direction?'.
I didn't explore the actual manufacturing process.