Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

emergingtrajectories.com

Post reply on HN
Print view View original

271–280 of 349 posts

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#271

Post by dymk »

hughw wrote: post #134
LarsDu88 wrote: post #48

The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest. The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release. Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software en…

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand. 640K ought to be enough for anybody.

I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#272

Post by hughw »

dymk wrote: post #271
hughw wrote: post #134

Earlier quoted context omitted.

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand. 640K ought to be enough for anybody.

I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?

probably no number you come up with will be adequate for very long. [edit] it's a reference to this possibly apocryphal prediction https://www.computerworld.com/article/1563853/the-640k-quote...

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#273

Post by throwawayffffas »

LarsDu88 wrote: post #48

The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest. The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release. Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software en…

> whoever burns their models to ASICs fastest.

There is already custom hardware see cerebras.

GPUs have a lot of slack there is at least one lab that had a (small 8b) model generate almost 3000 tokens per second on a MI300X for a talk, instead of the typical software stack that did maybe 100ish tokens per second.

High bandwidth flash storage is in the works, i.e hard drives with TBs of storage and over 1 TB per second of read speeds. Meaning that in a couple of years you may be able to buy a card with 40-90GBs of HBM and 4TB of HBF and run a 3T model locally at a reasonable speed for 10-20k as opposed to a cool mil.

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#274

Post by quotemstr »

Tuna-Fish wrote: post #259
akiselev wrote: post #161

Earlier quoted context omitted.

The weights are in SRAM, so the LLM architecture is burned in but the weights can be updated.

Taalas only uses SRAM for the KV cache and the activations, the weights are in mask rom in the metal layers. If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.

Is the mask ROM really going to be worth it over Carmack's high-bandwidth-flash concept? I mean, sure, I could be convinced I guess, but it's not obvious.

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#275

Post by oliyoung »

dymk wrote: post #271
hughw wrote: post #134

Earlier quoted context omitted.

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand. 640K ought to be enough for anybody.

I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?

The "640k should be enough for anyone" quote (even if Gates didn't exactly say it) is making the point is that _right now_ we have no idea about what our future needs and capabilities will be, we can't imagine what "should be enough" will be

640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#276

Post by boc »

yogthos wrote: post #234
boc wrote: post #213

Earlier quoted context omitted.

Or option three is they are drafting hard off the frontier US models via distillation.

The process takes time because even when you're distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn't been much time to do that. On top of that, Kimi also does better than Fable or GPT on a lot of tasks, distillation alone can't explain that, meaning there is a difference in architecture. You can watch a talk from Kimi…

> US companies models constantly distill each other as Musk was forced to admit under oath

> This whole narrative has just been a massive cope.

So wait, US AI companies all use distillation because... it's not effective and it's all just cope? Or is distillation really powerful and they all do it, which Musk was forced to admit under oath? But when China does distillation it isn't powerful and they don't need to do it, but they do it anyway because it's fun?

Either it's powerful and everyone, including the Chinese labs, use it as a way to rapidly catch-up against the SOTA models, or it's a red herring and the huge amounts of energy spent to protect and enable distillation is all just wasted money. Which is it?

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#277

Post by jfim »

hedgehog wrote: post #195
akiselev wrote: post #161

Earlier quoted context omitted.

The weights are in SRAM, so the LLM architecture is burned in but the weights can be updated.

No the weights are in the metal layers, they cannot be updated.

The base weights can't be updated but from what I recall it allows adding a low rank adapter to customize the model a little bit.

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#278

Post by achandra03 »

jesse_dot_id wrote: post #238
port3000 wrote: post #50

It's amazing how quickly Fable went from 'Game-changing model that needs to be banned' to 'Yeah it's alright, but OpenAI is also just as good and there are a couple of good open weight alternatives that are equivalent for almost everything' The hype cycles are shortening, perhaps we really are reaching some kind of plateau this time (famous last words)

The plateau is inevitable because their rapacious training methodologies are only viable when there are no defense in place, but information continues to evolve, which means the models will have to be continuously updated, but will be doing so with less and less freely available data.

> with less and less freely available data

My understanding is that the labs ran out of freely available data to train on a while ago, and now primarily rely on human data vendors such as Surge and Mercor to source their data.

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#279

Post by uncivilized »

cmrdporcupine wrote: post #264
uncivilized wrote: post #260

Earlier quoted context omitted.

A CUDA kernel, compiler optimization, or anything of similar complexity

Feel free to look, but don't bother replying if you have something negative to say, don't feel like having my day ruined by negativity. https://github.com/rdaum/mica/tree/main/crates/relation-kern... https://github.com/rdaum/mica/tree/main/crates/vm https://github.com/rdaum/pagebox/blob/main/crates/wal/src/wa...

Thanks for sharing. Looks interesting. Was all the code AI generated, or is it a mix?

Re: Kimi K3, Qwen 3.8, and Anthropic's (Potential) Unravelling

#280

Post by t0mpr1c3 »

jesse_dot_id wrote: post #238
port3000 wrote: post #50

It's amazing how quickly Fable went from 'Game-changing model that needs to be banned' to 'Yeah it's alright, but OpenAI is also just as good and there are a couple of good open weight alternatives that are equivalent for almost everything' The hype cycles are shortening, perhaps we really are reaching some kind of plateau this time (famous last words)

The plateau is inevitable because their rapacious training methodologies are only viable when there are no defense in place, but information continues to evolve, which means the models will have to be continuously updated, but will be doing so with less and less freely available data.

We are only just starting to model the physical world. There will be more training on empirical data.
Post reply on HN