Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Taalas “prints” LLM onto a chip?

anuragk.com

Post reply on HN
Print view View original

181–190 of 266 posts

Re: How Taalas “prints” LLM onto a chip?

#181

Post by yunohn »

RobotToaster wrote: post #118
the__alchemist wrote: post #100

Earlier quoted context omitted.

I believe this is a CPU/GPU vs ASIC comparison, rather than CPU vs GPU. They have always(ish) coexisted, being optimized for different things: ASICs have cost/speed/power advantages, but the design is more difficult than writing a computer program, and you can't reprogram them. Generally, you use an ASIC to perform a specific task. In this case, I think the takeaway is the LLM functionality here is performance-sensit…

It reminds me of the switch from GPUs to ASICs in bitcoin mining. I've been expecting this to happen.

But the BTC mining algorithm has not and will not change. That’s the only reason ASICs atleast make a bit of sense for crypto.

AI being static weights is already challenged with the frequent model updates we already see - but may even be a relic once we find a new architecture.

Re: How Taalas “prints” LLM onto a chip?

#182

Post by sowbug »

amelius wrote: post #86
thesz wrote: post #30

8B coefficients are packed into 53B transistors, 6.5 transistors per coefficient. Two-inputs NAND gate takes 4 transistors and register takes about the same. One coefficient gets processed (multiplied by and result added to a sum) with less than two two-inputs NAND gates. I think they used block quantization: one can enumerate all possible blocks for all (sorted) permutations of coefficients and for each layer place…

I'm looking forward to the model.toVHDL() method in PyTorch.

Ugh, quick, everyone start panic-buying FPGAs now.

Re: How Taalas “prints” LLM onto a chip?

#183

Post by MarsIronPI »

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

Doesn't Google have custom TPUs that are kind of a halfway point between Taalas' approach and a generic GPU? I wonder if that kind of hardware will reach consumers. It probably will, though as I understand them NPUs aren't quite it.

Re: How Taalas “prints” LLM onto a chip?

#184

Post by sowbug »

saati wrote: post #155
pwarner wrote: post #136

Earlier quoted context omitted.

I'd be kind of shocked if Nvidia isn't playing with this. I don't expect it's like super commercially viable today, but for sure things need to trend to radically more efficient AI solutions.

These are chips that become e-waste the second a better a model comes out, and nvidia is already limited by TSMC capacity.

They'll be perfect for an appliance like the Rick and Morty butter robot.

Re: How Taalas “prints” LLM onto a chip?

#185

Post by MarsIronPI »

HPsquared wrote: post #169
cyanydeez wrote: post #164

Earlier quoted context omitted.

Only in VC backed funding land. In the real world, theres talking refrigerators who dont need to know how to recite shakespeare.

On the upside, Shakespeare isn't going to change soon.

So you're saying we should burn Shakespeare onto a chip? /s

Re: How Taalas “prints” LLM onto a chip?

#186

Post by dyauspitr »

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

Job specific ASICs are are “old as time.”

Re: How Taalas “prints” LLM onto a chip?

#187

Post by MarsIronPI »

adeelk93 wrote: post #150
Aurornis wrote: post #137

Earlier quoted context omitted.

The die size is huge. This isn’t the kind of chip that would go into your MacBook, let alone an iPhone. It’s for cloud based servers.

And computers used to be the size of a room. I think they can get it to iPhone size in the future, this is an early prototype.

Well, there's a limit to how small we can make transistors with our current technology. As I understand it, Intel is already running into those limits with their new CPUs (they had to redesign the fins IIRC). I can imagine that without an actual breakthrough in chip manufacturing the size could stay large. That's not to say that a breakthrough won't happen, though.

Re: How Taalas “prints” LLM onto a chip?

#188

Post by sowbug »

MarcLore wrote: post #78

[dead]

As scary as it sounds today, a lightning-quick zero latency non-networked local LLM could provide value in an application like a self-driving car. It would be a level below Waymo's remote human support, so if the car couldn't figure out how to deal with a weird situation, it could ask the LLM what to do, hopefully avoiding the need to phone home (and perhaps handling cases where it couldn't phone home).

Re: How Taalas “prints” LLM onto a chip?

#190

Post by ivan_gammel »

ACCount37 wrote: post #173
ivan_gammel wrote: post #61

Earlier quoted context omitted.

It might be not that bad. “Good enough” open-weight models are almost there, the focus may shift to agentic workflows and effective prompting. The lifecycle of a model chip will be comparable to smartphones, getting longer and longer, with orchestration software being responsible for faster innovation cycles.

"Good enough" open weights models were "almost there" since 2022. I distrust the notion. The bar of "good enough" seems to be bolted to "like today's frontier models", and frontier model performance only ever goes up.

The generation of frontier models from H1 2025 is the good enough benchmark.
Post reply on HN