Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Taalas “prints” LLM onto a chip?

anuragk.com

Post reply on HN
Print view View original

121–130 of 266 posts

Re: How Taalas “prints” LLM onto a chip?

#121

Post by selcuka »

arisAlexis wrote: post #90
kioku wrote: post #82

I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?

and run an outdated model for 3 years while progress is exponential? what is the point of that

> what is the point of that

Planned obsolescence? /s

Jokes aside, they can make the "LLM chip" removable. I know almost nothing is replaceable in MacBooks, but this could be an exception.

Re: How Taalas “prints” LLM onto a chip?

#122

Post by post-it »

JKCalhoun wrote: post #109
byran wrote: post #3

This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc. Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies…

Apple should have done this yesterday. A local AI on my phone/Macbook is all I really want from this tech. The cloud-based AI (OpenAI, etc.) are todays AOL.

The hardware isn't there yet. Apple's neural engine is neat and has some uses but it just isn't in the same league as Claude right now. We'll get there.

Re: How Taalas “prints” LLM onto a chip?

#125

Post by androiddrew »

amelius wrote: post #86
thesz wrote: post #30

8B coefficients are packed into 53B transistors, 6.5 transistors per coefficient. Two-inputs NAND gate takes 4 transistors and register takes about the same. One coefficient gets processed (multiplied by and result added to a sum) with less than two two-inputs NAND gates. I think they used block quantization: one can enumerate all possible blocks for all (sorted) permutations of coefficients and for each layer place…

I'm looking forward to the model.toVHDL() method in PyTorch.

Is this a thing?

Re: How Taalas “prints” LLM onto a chip?

#127

Post by ramshanker »

I can imagine, where this becomes a mainstream PCIe extension card. Like back in days we had separate graphics card, audio card etc. Now AI card. So to upgrade the PC to latest model, we could buy a new card, load up the drivers and boom, intelligence upgrade of the PC. This would be so cool.
Post reply on HN