Earlier quoted context omitted.
So, like the present
With more wealth concentrates at the top, yes.
Like some kind of pimple, which we can pop from all sides. And then we'll build something different. Something that works.
AMD acquires Taalas to boost inference performance by etching models in silicon
341–350 of 712 posts
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#342Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.
Feels both unreal and dystopian. The speed at which these models are developing is very scary
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#343What I like about this, is that it significantly increases the probability of a sci-fi scenario where you're picking up a hot chip on the black market; rumor has it, Mythos 9 weights baked in...
Plug it in, and it's a old prototype with Gemma 5 weights baked onboard. Dammit, fucked by Craigslist again!
Back in the kazaa and limewire days, you'd sometimes try to get a movie / episode from a series, wait hours / days for it to download, and when it was done you had a ~50/50 chance to actually watch what you wanted or an old german porn movie :/
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#344I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition. Baking models onto silicon would've been the next logical move to get a moat. Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.
This seems like a very bad and dangerous direction for our society.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#345> At 20 billion parameters per chip, you’d need just 50 accelerators to support a trillion-parameter model I don't see any evidence that this is possible. From my understanding, the whole model needs to be on a single chip. Which rules out any popular frontier models with several trillions of parameters. Even smaller sub-frontier models have hundreds of millions of parameters, so these would be ruled out as well.
They pipeline-parallelize across multiple chips. DeepSeek v4 Pro will be 30 chips.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#346Earlier quoted context omitted.
I'm not sure inference speed is always the slowest thing for me right now. The agent is running tests, loading webpages, etc, which all take time. I don't know if a fast agent would speed things up in all cases. That said, it obviously depends on the project.
> " The agent is running tests, loading webpages, etc, which all take time " A frustrating vision of the future would be when we've been asking for faster loading lighter web pages for years and then companies start caring about it and improving it not for us humans but for LLMs.
That happened at corpo work for each of:
* Build times
* CI latency
* Developer tooling
* Documentation
* Modularity
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#347Question: we currently emulate neural networks by performing matrix math in synchronous clock CPU architectures. Would it not be better to abandon synchronization and etch neuron synapses directly in silicon, keeping only the weights variable? I think some researchers are pursuing this, but I forget what the approach is called.
"Neuromorphic chips" .... and I have the exact same question in mind.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#348Earlier quoted context omitted.
It is my understanding that just baking the model itself into silicon only gives moderate gains because memory bandwidth remains a bottleneck.
The big benefit is ROM cells require fewer components than DRAM. So the chips would be tiny, dense, cheap and consume far less power.
I thought DRAM was pretty dense already. Is mask ROM that much denser?
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#349Earlier quoted context omitted.
I don’t get why this is an issue? You can run Claude/OpenAI SOTA models through Amazon bedrock. These weights have to live somewhere to run on Bedrock.
somewhere = an AWS data center with multiple layers of security and NDAs They won't sell/rent/license the weights to an end user at any price because they don't trust your security.
I work in embedded space. Just because it’s in hardware doesn’t mean you can’t “protect” it. Most modern software (regardless if it’s hardware or not) can be cryptophically signed.
Re: AMD acquires Taalas to boost inference performance by etching models in silicon
#350Earlier quoted context omitted.
The big benefit is ROM cells require fewer components than DRAM. So the chips would be tiny, dense, cheap and consume far less power.
I thought DRAM was pretty dense already. Is mask ROM that much denser?
Yes, each rom bit can be a transistor or even a diode with a decoder circuit. Simplest Dram cell is capacitor+transistor - and you need a clock, refresh circuit etc.
Someday, I imagine model weights could even be encoded as analog resistors (memristors or similar) for even greater density