Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

wafer.ai

Post reply on HN
Print view View original

31–40 of 119 posts

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#31

Post by throwa356262 »

villgax wrote: post #7

Lol, such a lazily written article by wafer.ai GPUs. 8× MI355X (TP8) B300 (TP8+DCP8) Decode tok/s per stream 118 tok/s 172 tok/s Peak aggregate. 952 tok/s 1,568 tok/s Peak aggregate per GPU 119 tok/s 196 tok/s On every row the B300 beat the MI355X The B200 is being forcefully compared against something which is not gonna fit within it's memory in a single node & not much details about multi-node interconnectivity, di…

Did you see this section?

    To the B200’s defence, its numbers are somewhat deflated by the fact that it pays a cross-node all-reduce on the decode critical path (RoCE v2 at ~195 Gb/s) — it’s the only config here that spans two nodes

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#33

Post by gpugreg »

croemer wrote: post #30
swiftcoder wrote: post #27

Earlier quoted context omitted.

Even though none of us could actually afford to train it?

Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper. Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.

> Some of us might be rich

I sure wish I had a few 100M of disposable income to train a frontier model.

> or in the future it could be useful when training is cheaper.

I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#34

Post by inferencecoder »

logicallee wrote: post #15
inferencecoder wrote: post #8

Earlier quoted context omitted.

There is barely any effort, a simple GPT5.6 sol pro query rips the post apart.

if you did that in the web interface, could you share the chat? I'd be interested to read it.

https://chatgpt.com/share/6a6f09ff-2830-83ea-9578-de3016cfae...

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#35

Post by Muromec »

gpugreg wrote: post #33
croemer wrote: post #30

Earlier quoted context omitted.

Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper. Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.

> Some of us might be rich I sure wish I had a few 100M of disposable income to train a frontier model. > or in the future it could be useful when training is cheaper. I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#36

Post by Muromec »

ciupicri wrote: post #25
Alpha3031 wrote: post #17

Earlier quoted context omitted.

Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source. I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point…

You can also edit/patch binary executables, but no one says about that software that is open source.

You can, but its not the first preference

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#37

Post by gpugreg »

Muromec wrote: post #35
gpugreg wrote: post #33

Earlier quoted context omitted.

> Some of us might be rich I sure wish I had a few 100M of disposable income to train a frontier model. > or in the future it could be useful when training is cheaper. I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful.

Looks like global energy consumption has risen by an order of magnitude from 1900 to 2000: https://www.encyclopedie-energie.org/en/world-energy-consump...

Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#38

Post by mlazos »

kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#39

Post by WithinReason »

croemer wrote: post #26
WithinReason wrote: post #20

Earlier quoted context omitted.

If they open sourced the training data and code but you had to train the network yourself, would that be open source?

Yes, obviously.

As an example, Grok 4 took $388M to train

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#40

Post by andy99 »

mlazos wrote: post #38
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

> obsession with precise terms

They’re not really even being precise. The relevant software freedom, from the FSF is []

  The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.

Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.

[] https://www.gnu.org/philosophy/free-sw.html#four-freedoms

Post reply on HN