Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Taalas “prints” LLM onto a chip?

anuragk.com

Post reply on HN
Print view View original

101–110 of 266 posts

Re: How Taalas “prints” LLM onto a chip?

#101

Post by IshKebab »

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

> Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics.

We transitioned from software on CPUs to fixed GPU hardware... But then we transitioned back to software running on GPUs! So there's no way you can say "of course this is the future".

Re: How Taalas “prints” LLM onto a chip?

#102

Post by IshKebab »

roncesvalles wrote: post #19
byran wrote: post #3

This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc. Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies…

Well even programmable ASICs like Cerebras and Groq give many-multiples speedup over GPUs and the market has hardly reacted at all.

Cerebras gives a many multiple speedup but it's also many multiples more expensive.

Re: How Taalas “prints” LLM onto a chip?

#103

Post by qoez »

> It took them two months, to develop chip for Llama 3.1 8B. In the AI world where one week is a year, it's super slow. But in a world of custom chips, this is supposed to be insanely fast.

LLama 3.1 is like 2 years at this point. Taking two months to convert a model that only updates every 2 years is very fast

Re: How Taalas “prints” LLM onto a chip?

#104

Post by padjo »

arisAlexis wrote: post #90
kioku wrote: post #82

I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?

and run an outdated model for 3 years while progress is exponential? what is the point of that

Is progress still exponential? Feels like its flattening to me, it is hard to quantify but if you could get Opus 4.2 to work at the speed of the Taalas demo and run locally I feel like I'd get an awful lot done.

Re: How Taalas “prints” LLM onto a chip?

#105

Post by grosswait »

lm28469 wrote: post #72
lancebeet wrote: post #64

Earlier quoted context omitted.

You obviously don't believe that AGI is coming in two release cycles, and you also don't seem to have much faith in the new models containing massive improvements over the last ones. So the answer to who is going to pay for these custom chips seems to be you.

Why would I buy chips to run handicapped models when the 10+ llms players all offer free tier access to their 1t+ parameters models ?

Do you think the free gravy train will run forever?

Re: How Taalas “prints” LLM onto a chip?

#106

Post by amelius »

Hendrikto wrote: post #89
amelius wrote: post #79

Earlier quoted context omitted.

> USB-C With these speeds you can run it over USB2, though maybe power is limiting.

USB-C is just a form factor and has nothing to do with which protocol you run at which speeds.

I wasn't talking about the form factor.

Re: How Taalas “prints” LLM onto a chip?

#107

Post by GTP »

the__alchemist wrote: post #100
bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

I believe this is a CPU/GPU vs ASIC comparison, rather than CPU vs GPU. They have always(ish) coexisted, being optimized for different things: ASICs have cost/speed/power advantages, but the design is more difficult than writing a computer program, and you can't reprogram them. Generally, you use an ASIC to perform a specific task. In this case, I think the takeaway is the LLM functionality here is performance-sensit…

The middle ground here would be an FPGA, but I belive you would need a very expensive one to implement an LLM on it.

Re: How Taalas “prints” LLM onto a chip?

#108

Post by JKCalhoun »

bsenftner wrote: post #93

I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…

"This has been demonstrated already…"

I think burning the weights into the gates is kinda new.

("Weights to gates." "Weighted gates"? "Gated weights"?)

Re: How Taalas “prints” LLM onto a chip?

#109

Post by JKCalhoun »

byran wrote: post #3

This would be a very interesting future. I can imagine Gemma 5 Mini running locally on hardware, or a hard-coded "AI core" like an ALU or media processor that supports particular encoding mechanisms like H.264, AV1, etc. Other than the obvious costs (but Taalas seems to be bringing back the structured ASIC era so costs shouldn't be that low [1]), I'm curious why this isn't getting much attention from larger companies…

Apple should have done this yesterday. A local AI on my phone/Macbook is all I really want from this tech.

The cloud-based AI (OpenAI, etc.) are todays AOL.

Re: How Taalas “prints” LLM onto a chip?

#110

Post by GTP »

amelius wrote: post #79
roncesvalles wrote: post #20

Earlier quoted context omitted.

That slot is called USB-C. I can fully imagine inference ASICs coming in powerbank form factor that you'd just plug and play.

> USB-C With these speeds you can run it over USB2, though maybe power is limiting.

You would likely need external power anyway.
Post reply on HN