How Taalas “prints” LLM onto a chip? • Hacker News Board

Re: How Taalas “prints” LLM onto a chip?

#111

Post by yonatan8070 » Sun, Feb 22, 2026, 1:20 PM UTC

dmurray wrote: post #31

XorNot wrote: post #24

Earlier quoted context omitted.

Pretty sure it'd just be a thumbdrive. Are the Taalas chips particularly large in surface area?

The only product they've announced at the moment [0] is a PCI-e card. It's more like a small power bank than a big thumb drive. But sure, the next generation could be much smaller. It doesn't require battery cells, (much) heat management, or ruggedization, all of which put hard limits on how much you can miniaturise power banks. [0] https://taalas.com/the-path-to-ubiquitous-ai/

I wouldn't call that size a small power bank. That chip is in the same ballpark as gaming GPUs, and based on the VRMs in the picture it probably draws about as much power.

But as you said, the next generations are very likely to shrink (especially with them saying they want to do top of the line models in 2 generations), and with architecture improvements it could probably get much smaller.

Re: How Taalas “prints” LLM onto a chip?

#112

Post by yonatan8070 » Sun, Feb 22, 2026, 1:27 PM UTC

NitpickLawyer wrote: post #68

beAroundHere wrote: post #6

Earlier quoted context omitted.

That's the kind of hardware am rooting for. Since it'll encourage Open weighs models, and would be much more private. Infact, I was thinking, if robots of future could have such slots, where they can use different models, depending on the task they're given. Like a Hardware MoE.

> Since it'll encourage Open weighs models Is this accurate? I don't know enough about hardware, but perhaps someone could clarify: how hard would it be to reverse engineer this to "leak" the model weights? Is it even possible? There are some labs that sell access to their models (mistral, cohere, etc) without having their models open. I could see a world where more companies can do this if this turns out to be a via…

It's likely possible to extract model weights from the chip's design, but you'd need tooling at the level of an Intel R&D lab, not something any hobbyist could afford.

I doubt anyone would have the skills, wallet, and tools to RE one of these and extract model weights to run them on other hardware. Maybe state actors like the Chinese government or similar could pull that off.

Re: How Taalas “prints” LLM onto a chip?

#113

Post by dogma1138 » Sun, Feb 22, 2026, 1:41 PM UTC

GTP wrote: post #107

the__alchemist wrote: post #100

Earlier quoted context omitted.

I believe this is a CPU/GPU vs ASIC comparison, rather than CPU vs GPU. They have always(ish) coexisted, being optimized for different things: ASICs have cost/speed/power advantages, but the design is more difficult than writing a computer program, and you can't reprogram them. Generally, you use an ASIC to perform a specific task. In this case, I think the takeaway is the LLM functionality here is performance-sensit…

The middle ground here would be an FPGA, but I belive you would need a very expensive one to implement an LLM on it.

FPGAs would be less efficient than GPUs.

FPGAs don’t scale if they did all GPUs would’ve been replaced by FPGAs for graphics a long time ago.

You use an FPGA when spinning a custom ASIC doesn’t makes financial sense and generic processor such as a CPU or GPU is overkill.

Arguably the middle ground here are TPUs, just taking the most efficient parts of a “GPU” when it comes to these workloads but still relying on memory access in every step of the computation.

Re: How Taalas “prints” LLM onto a chip?

#114

Post by mips_avatar » Sun, Feb 22, 2026, 1:45 PM UTC

roncesvalles wrote: post #19

byran wrote: post #3

This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc. Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies…

Well even programmable ASICs like Cerebras and Groq give many-multiples speedup over GPUs and the market has hardly reacted at all.

The problem with groq was they only allowed LORA on llama 8b and 70b, and you had to have an enterprise contract it wasn't self service.

Re: How Taalas “prints” LLM onto a chip?

#115

Post by mips_avatar » Sun, Feb 22, 2026, 1:46 PM UTC

xattt wrote: post #83

kioku wrote: post #82

I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?

Is it possible to supplement the model with a diff for updates on modular memory, or would severely impact perf?

I imagine you could do something like a LORA

Re: How Taalas “prints” LLM onto a chip?

#116

Post by dogma1138 » Sun, Feb 22, 2026, 1:47 PM UTC

JKCalhoun wrote: post #108

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

"This has been demonstrated already…" I think burning the weights into the gates is kinda new. ("Weights to gates." "Weighted gates"? "Gated weights"?)

Not really new, this is 80’s-90’s Neuron MOS Transistor.

It’s also not that different than how TPUs work where they have special registers in their PEs for weights.

Re: How Taalas “prints” LLM onto a chip?

#117

Post by bagful » Sun, Feb 22, 2026, 2:04 PM UTC

roncesvalles wrote: post #20

owenpalmer wrote: post #5

> Kinda like a CD-ROM/Game cartridge, or a printed book, it only holds one model and cannot be rewritten. Imagine a slot on your computer where you physically pop out and replace the chip with different models, sort of like a Nintendo DS.

That slot is called USB-C. I can fully imagine inference ASICs coming in powerbank form factor that you'd just plug and play.

Like the chip-software in Gibson’s sprawl, from the micro-soft to the ROM cowboy to the Aleph, the endgame of computertool distribution is via single-use chunks of quasi-biological computronium

Re: How Taalas “prints” LLM onto a chip?

#118

Post by RobotToaster » Sun, Feb 22, 2026, 2:05 PM UTC

the__alchemist wrote: post #100

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

I believe this is a CPU/GPU vs ASIC comparison, rather than CPU vs GPU. They have always(ish) coexisted, being optimized for different things: ASICs have cost/speed/power advantages, but the design is more difficult than writing a computer program, and you can't reprogram them. Generally, you use an ASIC to perform a specific task. In this case, I think the takeaway is the LLM functionality here is performance-sensit…

It reminds me of the switch from GPUs to ASICs in bitcoin mining. I've been expecting this to happen.

Re: How Taalas “prints” LLM onto a chip?

#119

Post by cedws » Sun, Feb 22, 2026, 2:16 PM UTC

MarcLore wrote: post #78

[dead]

The network latency bit deserves more attention. I’ve been trying to find out where AI companies are physically serving LLMs from but it’s difficult to find information about this. If I’m sitting in London and use Claude, where are the requests actually being served?

The ideal world would be an edge network like Cloudflare for LLMs so a nearby POP serves your requests. I’m not sure how viable this is. On classic hardware I think it would require massive infra buildout, but maybe ASICs could be the key to making this viable.

Re: How Taalas “prints” LLM onto a chip?

#120

Post by RobotToaster » Sun, Feb 22, 2026, 2:18 PM UTC

byran wrote: post #3

This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc. Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies…

> I'm curious why this isn't getting much attention from larger companies.

I can see two potential reasons:

1) Most of the big players seem convinced that AI is going to continue to improve at the rate it did in 2025, if their assumption is somehow correct by the time any chip entered mass production it would be obsolete.

2) The business model of the big players is to sell expensive subscriptions, and train on and sell the data you give it. Chips that allow for relatively inexpensive offline AI aren't conducive to that.