Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD acquires Taalas to boost inference performance by etching models in silicon

theregister.com

Post reply on HN
Print view View original

461–470 of 712 posts

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#461

Post by Ericson2314 »

manmal wrote: post #388
QuiDortDine wrote: post #251

Earlier quoted context omitted.

Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model. Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become…

Errors compound, and making 1000 wrong decisions per hour, will not result in something useful. Maybe you‘ve tried setting up guardrails for good design or architecture at some point? I think it’s simply not possible to do that. It would certainly be an accelerator for people who know exactly what they want. And it would remove multi tasking, which I‘d appreciate.

If your task has incremental rewards/feedback, you can push the "intelligence rate" simply by sampling the reward function faster. That's not fake, even if it not a substitute either.

This is the "dumber but honest person that works harder" phenomenon, vs "lazy genius".

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#462

Post by gf000 »

dzhiurgis wrote: post #437
fsloth wrote: post #419

Earlier quoted context omitted.

I don’t think average user _needs_ to solve frontier challenges. ”Call to Jane”, ”turn on the lights” and ”what’s the weather this afternoon” is more like it I would guess. Ofc if the model has some critical bugs that’s another matter.

Your examples worked on phones for over a decade. Maybe baking in a model that is "certified" to have some unconditioned truths + rest is pulled from external models/store could make sense. But AFAIK that doesn't exist and I'm not sure it can possibly be made. Perhaps society as a whole at least can work on an open corpus of training data, but I'm not holding my breath on this.

It barely works even today, like Siri is laughably bad.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#463

Post by TeMPOraL »

lelanthran wrote: post #414
adrianN wrote: post #332

Earlier quoted context omitted.

It is my understanding that just baking the model itself into silicon only gives moderate gains because memory bandwidth remains a bottleneck.

Did you try using the the talaas chat? Something stupid like 18k tokens/second. Think it's called Askjimmy or similar.

What is the model they're using there though? Interrogated, it claims it's a BERT variant and has capabilities around GPT-3 and below GPT-4.

(Not that I believe it, it writes too well for GPT-3.)

Hosted frontier models from two years ago would be much faster today, too.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#464

Post by rldjbpin »

given their specs [1] show ~100 mm^2 of silicon for each billion params (albeit in 6 nm process), what will come first: nuclear fusion or one of these that can handle the multi-trillon param models of today?

[1] https://taalas.com/products/

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#465

Post by akie »

rldjbpin wrote: post #464

given their specs [1] show ~100 mm^2 of silicon for each billion params (albeit in 6 nm process), what will come first: nuclear fusion or one of these that can handle the multi-trillon param models of today? [1] https://taalas.com/products/

You're assuming no improvement on their current technology

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#466

Post by gf000 »

Ericson2314 wrote: post #461
manmal wrote: post #388

Earlier quoted context omitted.

Errors compound, and making 1000 wrong decisions per hour, will not result in something useful. Maybe you‘ve tried setting up guardrails for good design or architecture at some point? I think it’s simply not possible to do that. It would certainly be an accelerator for people who know exactly what they want. And it would remove multi tasking, which I‘d appreciate.

If your task has incremental rewards/feedback, you can push the "intelligence rate" simply by sampling the reward function faster. That's not fake, even if it not a substitute either. This is the "dumber but honest person that works harder" phenomenon, vs "lazy genius".

That's a good way to put it, but still my experience is that worse code bases are non-linearly harder to maintain and improve in the future, software tends to break down without a good enough base.

Sure, in the future full rewrites and stuff like that will be just another "throw money at it" problem, but fundamentally software can get arbitrary complex and we barely know how to write large, maintainable code bases.

Nonetheless, I think testing (and maybe proofs) will have its long-awaited time to shine, as being the "reward function".

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#467

Post by dgellow »

Certhas wrote: post #436
ldng wrote: post #434

Earlier quoted context omitted.

How ? Do LLMs actually "know' when they don't "know" ?

How do humans?

Always the same trick of not answering the question and deflecting to „what about humans“. Can you folks not evaluate LLMs as the system they are, without vague gestures at how a different system behaves?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#468

Post by dave1010uk »

I'm surprised there's not more discussion about potential inflection points here. When technology gets faster, it opens up whole new classes of UX that were hard to predict

For example, faster internet didn't mean being able to view 100x as many HTML4 web pages. It brought SaaS, streaming media and interactivity.

I'm not good at predicting, but some ideas:

1. All information gets augmented in real time with personalised context.

2. AI interaction seems more like find-as-you-type than a back and forth.

3. AI produces many outputs to pick from. Either the human, AI, or another system can do the deciding.

Even if it's last year's model, speeding up LLM inference could open up all sorts of opportunities.

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#469

Post by Perz1val »

hliyan wrote: post #305

Question: we currently emulate neural networks by performing matrix math in synchronous clock CPU architectures. Would it not be better to abandon synchronization and etch neuron synapses directly in silicon, keeping only the weights variable? I think some researchers are pursuing this, but I forget what the approach is called.

That'd be either analog computing or you're implementing floating point number math in each neuron. Maybe some kind of PWM calculations would be doable, do we have it researched or would be a new kind of math?

Re: AMD acquires Taalas to boost inference performance by etching models in silicon

#470

Post by whazor »

rldjbpin wrote: post #464

given their specs [1] show ~100 mm^2 of silicon for each billion params (albeit in 6 nm process), what will come first: nuclear fusion or one of these that can handle the multi-trillon param models of today? [1] https://taalas.com/products/

According to my agent:

> At 6 nm: 1T doesn't fit on one wafer.

> At ~2 nm: 1T plausibly fits comfortably on one 300 mm wafer.

But then again, 300B to 500B models are to this day also very valuable

Post reply on HN