Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

431–440 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#431

Post by planb »

LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

They are: https://openai.com/index/cerebras-partnership/ My guess is they only consider Luna "good enough" to justify the immense up-front investment to put it onto silicon, but Luna at 10x the current speed would be killer. If they're really pursuing live voice conversations with a hardware assistant, latency is more important than accuracy (for complex questions the assistant could always say something like "wait a minute, I need to think about this" and hand over to another model).

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#432

Post by askvictor »

throwaway240403 wrote: post #203
askvictor wrote: post #105

Earlier quoted context omitted.

People already buy new phones every year, this just creates even more reason to do so

Your location/income bias is showing. Most people do not buy new phones every year.

I never said most people. But it's not uncommon. I personally find it ridiculous, and hold onto my phone until it's unusable, but plenty of people in middle class Australia seem convinced that they need the new one whenever it comes out.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#433

Post by jjcm »

tasty_freeze wrote: post #298
la6479 wrote: post #214

Earlier quoted context omitted.

Just to see how fast it is try chatjimmy.ai

It is really fast and ... really hallucinates. I asked "Does the Wang corporation still exist? If not, what happened to it?" and it replied (in part): "Yes, the Wang Corporation, the company that originally developed and marketed the Wang 2200 computer, still exists as a rebranded company under the name PPL (Precision Pencil and Label), but it has undergone significant changes and challenges over the years. Here's a…

to be fair, it's running an 8b model from like 2 years ago. Taalas just does the chip design, not the model architecture.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#434

Post by ldng »

planb wrote: post #431
LarsDu88 wrote: post #44

I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

They are: https://openai.com/index/cerebras-partnership/ My guess is they only consider Luna "good enough" to justify the immense up-front investment to put it onto silicon, but Luna at 10x the current speed would be killer. If they're really pursuing live voice conversations with a hardware assistant, latency is more important than accuracy (for complex questions the assistant could always say something like "wait a…

How ? Do LLMs actually "know' when they don't "know" ?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#435

Post by selcuka »

IOT_Apprentice wrote: post #409
QuiDortDine wrote: post #251

Earlier quoted context omitted.

Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…

I tried it. I asked where Bruce Lee was born. It stated he was born in Hong Kong. I challenged it and it went further naming a hospital there. I stated he was born in San Francisco and it apologized and then said his father was a missionary traveling in America, which was also wrong. Bruce’s father was a famous Cantonese Opera singer and actor. This model had zero information right, while being fast in responding. Un…

It gave the correct answers to both questions for me:

> Bruce Lee was born in San Francisco, California, USA on November 27, 1940.

> Bruce Lee's father was a Chinese opera singer

That being said, this is not a good test. It is a language model (a very small one), not an encyclopedia.

ChatJimmy interface is just a tech demo. Without tool calling functionality we can't expect it to be factually correct.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#436

Post by Certhas »

ldng wrote: post #434
planb wrote: post #431

Earlier quoted context omitted.

They are: https://openai.com/index/cerebras-partnership/ My guess is they only consider Luna "good enough" to justify the immense up-front investment to put it onto silicon, but Luna at 10x the current speed would be killer. If they're really pursuing live voice conversations with a hardware assistant, latency is more important than accuracy (for complex questions the assistant could always say something like "wait a…

How ? Do LLMs actually "know' when they don't "know" ?

How do humans?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#437

Post by dzhiurgis »

fsloth wrote: post #419
onion2k wrote: post #387

Earlier quoted context omitted.

Baking the base models on to ROM makes a lot of economic sense. Less so for consumers though, because it'd mean the phone is out of date in 3 months when a better model comes along.

I don’t think average user _needs_ to solve frontier challenges. ”Call to Jane”, ”turn on the lights” and ”what’s the weather this afternoon” is more like it I would guess. Ofc if the model has some critical bugs that’s another matter.

Your examples worked on phones for over a decade.

Maybe baking in a model that is "certified" to have some unconditioned truths + rest is pulled from external models/store could make sense. But AFAIK that doesn't exist and I'm not sure it can possibly be made. Perhaps society as a whole at least can work on an open corpus of training data, but I'm not holding my breath on this.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#438

Post by planb »

ldng wrote: post #434
planb wrote: post #431

Earlier quoted context omitted.

They are: https://openai.com/index/cerebras-partnership/ My guess is they only consider Luna "good enough" to justify the immense up-front investment to put it onto silicon, but Luna at 10x the current speed would be killer. If they're really pursuing live voice conversations with a hardware assistant, latency is more important than accuracy (for complex questions the assistant could always say something like "wait a…

How ? Do LLMs actually "know' when they don't "know" ?

They do this all the time, I'm using ChatGPT in Instant mode and it auto updates to thinking if my question is complex. Most of the time this works.

To answer your question: A large language model itself does not know this (afaik). But chatbots are not "just LLMs" but a whole bunch of systems (and models) around them.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#439

Post by sroerick »

pastel8739 wrote: post #363
QuiDortDine wrote: post #251

Earlier quoted context omitted.

Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…

This is the same pitch that people make about AI today. Speed isn’t the differentiator, quality is

They are both the differentiator.

AI previously provided speed but not quality. As soon as quality reached an acceptable threshold, the speed became the reigning factor.

In my opinion the quality is still much lower, but speed means the cost is significantly lower also.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#440

Post by christkv »

There is a big risk in etching a model into silicon like this. We are still evolving what small models look like and improving their performance. When do you decide to etch one into silicon knowing that right now an improved one can be 3 months away.
Post reply on HN