I would appreciate some clarification on the "store 4 bits of data with one transistor" part.
This doesn't sound remotely possible, but I am here to be convinced.
How Taalas “prints” LLM onto a chip?
91–100 of 266 posts
Re: How Taalas “prints” LLM onto a chip?
#92Note that this doesn't answer the question in the title, it merely asks it.
Frankly the most critical question is if they can really take shortcuts on DV etc, which are the main reasons nobody else tapes out new chips for every model. Note that their current architecture only allows some LORA-Adapter based fine-tuning, even a model with an updated cutoff date would require new masks etc. Which is kind of insane, but props to them if they can make it work. From some announcements 2 years ago,…
TTS, speech recognition, ocr/document parsing, Vision-language-action models, vehicle control, things like that do seem to be the ideal applications. Latency constraints limit the utility of larger models in many applications.
Re: How Taalas “prints” LLM onto a chip?
#93I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attention to what we have, what we do to have what we have.
Re: How Taalas “prints” LLM onto a chip?
#94I would appreciate some clarification on the "store 4 bits of data with one transistor" part. This doesn't sound remotely possible, but I am here to be convinced.
They declined to say: https://www.eetimes.com/taalas-specializes-to-extremes-for-e...
Except they say it's fully digital, so not an analog multiplier
Re: How Taalas “prints” LLM onto a chip?
#95I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?
and run an outdated model for 3 years while progress is exponential? what is the point of that
It won’t stay exponential forever.
Re: How Taalas “prints” LLM onto a chip?
#96Earlier quoted context omitted.
You obviously don't believe that AGI is coming in two release cycles, and you also don't seem to have much faith in the new models containing massive improvements over the last ones. So the answer to who is going to pay for these custom chips seems to be you.
Why would I buy chips to run handicapped models when the 10+ llms players all offer free tier access to their 1t+ parameters models ?
Not all applications are chatbots. Many potential uses for LLMs/VLAMs are latency constrained.
Re: How Taalas “prints” LLM onto a chip?
#97I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?
and run an outdated model for 3 years while progress is exponential? what is the point of that
Yeah, the space moves so quickly that I would not want to couple the hardware with a model that might be outdated in a month. There are some interesting talking points but a general purpose programmable asic makes more sense to me.
Re: How Taalas “prints” LLM onto a chip?
#98[dead]
latency and control, and reliability of bandwidth and associated costs - however this isn't just the pull for specialised hardware but for local computing in general, specialised hardware is just the most extreme form of it
there are tasks that inherently benefit from being centralised away, like say coordination of peers across a large area - and there are tasks that strongly benefit from being as close to the user as possible, like low latency tasks and privacy/control-centred tasks
simultaneously, there's an overlapping pull to either side caused by the monetary interests of corporations vs users - corporations want as much as possible under their control, esp. when it's monetisable information but most things are at volume, and users want to be the sole controller of products esp. when they pay for them
we had dumb terminals already being pushed in the 1960s, the "cloud", "edge computing" and all forms of consolidation vs segregation periods across the industry, it's not going to stop because there's money to be made from the inherent advantages of those models and even the industry leaders cannot prevent these advantages from getting exploited by specialist incumbents
once leaders consolidates, inevitably they seek to maximise profit and in doing so they lower the barrier for new alternatives
ultimately I think the market will never stop demanding just having your own *** computer under your control and hopefully own it, and only the removal of this option will stop this demand; while businesses will never stop trying to control your computing, and providing real advantages in exchange for that, only to enter cycles of pushing for growing profitability to the point average users keep going back and forth
Re: How Taalas “prints” LLM onto a chip?
#99I’m just wondering how this translates to computer manufacturers like Apple. Could we have these kinds of chips built directly into computers within three years? With insanely fast, local on-demand performance comparable to today’s models?
and run an outdated model for 3 years while progress is exponential? what is the point of that
When output is good enough, other considerations become more important. Most people on this planet cannot afford even an AI subscription, and cost of tokens is prohibitive to many low margin businesses. Privacy and personalization matter too, data sovereignty is a hot topic. Besides, we already see how focus has shifted to orchestration, which can be done on CPU and is cheap - software optimizations may compensate hardware deficiencies, so it’s not going to be frozen. I think the market for local hardware inference is bigger than for clouds, and it’s going to repeat Android vs iOS story.
Re: How Taalas “prints” LLM onto a chip?
#100I'm surprised people are surprised. Of course this is possible, and of course this is the future. This has been demonstrated already: why do you think we even have GPUs at all?! Because we did this exact same transition from running in software to largely running in hardware for all 2D and 3D Computer Graphics. And these LLMs are practically the same math, it's all just obvious and inevitable, if you're paying attent…
I believe this is a CPU/GPU vs ASIC comparison, rather than CPU vs GPU. They have always(ish) coexisted, being optimized for different things: ASICs have cost/speed/power advantages, but the design is more difficult than writing a computer program, and you can't reprogram them.
Generally, you use an ASIC to perform a specific task. In this case, I think the takeaway is the LLM functionality here is performance-sensitive, and has enough utility as-is to choose ASIC.