Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

wafer.ai

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#41

Post by nextaccountic »

mlazos wrote: post #38
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about.

The weights are the output of a program, it's a binary. It's not source.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#42

Post by logicallee »

inferencecoder wrote: post #34
logicallee wrote: post #15

Earlier quoted context omitted.

if you did that in the web interface, could you share the chat? I'd be interested to read it.

https://chatgpt.com/share/6a6f09ff-2830-83ea-9578-de3016cfae...

thanks for sharing.

When I read your original comment, I was thinking you had just asked it to evaluate the article. (Like just "evaluate this article" or something.)

I don't think anything anyone (or any AI) has ever written or published (including Sol itself) wouldn't be torn apart by the prompt you gave though.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#43

Post by HPsquared »

WithinReason wrote: post #39
croemer wrote: post #26

Earlier quoted context omitted.

Yes, obviously.

As an example, Grok 4 took $388M to train

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights.

The weights are literally a binary blob.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#44

Post by HPsquared »

mlazos wrote: post #38
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

Model weights are literally a compressed blob of binary data. The end result of a compilation process.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#45

Post by inferencecoder »

logicallee wrote: post #42
inferencecoder wrote: post #34

Earlier quoted context omitted.

https://chatgpt.com/share/6a6f09ff-2830-83ea-9578-de3016cfae...

thanks for sharing. When I read your original comment, I was thinking you had just asked it to evaluate the article. (Like just "evaluate this article" or something.) I don't think anything anyone (or any AI) has ever written or published (including Sol itself) wouldn't be torn apart by the prompt you gave though.

I think it's fair to expect an extensive review of an article before publishing. Not everything has a set of serious flaws.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#46

Post by WithinReason »

HPsquared wrote: post #43
WithinReason wrote: post #39

Earlier quoted context omitted.

As an example, Grok 4 took $388M to train

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights. The weights are literally a binary blob.

The weights are the modifiable representation. You modify them with gradient updates.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#47

Post by j-bos »

andy99 wrote: post #40
mlazos wrote: post #38

Earlier quoted context omitted.

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

> obsession with precise terms They’re not really even being precise. The relevant software freedom, from the FSF is [] The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this. Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training d…

Agreed, The spirit is fulfilled, as evidenced by the massive ongoing development of, effectively, new weights grown out of the sourced weights,

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#48

Post by Almondsetat »

kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

Sorry, where exactly is "data" in "source"?

I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#49

Post by shock »

mlazos wrote: post #38
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

> Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.

There is no precise thinking without precise terms.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#50

Post by springtimesun »

logicallee wrote: post #3

This part sounds like AI assisted setting this up and benchmarking it: >The fix was trivially simple: zero-pad the head count 12→16, run the fast kernel, and extract the real 12 heads from the output. I've recently used a frontier AI (ChatGPT 5.6 Sol on ultra) to set up a much smaller local model, and the performance optimizations it introduced left the model totally incoherent. (The model just repeats a single chara…

I have never used sol, but I regularly use Claude to set up and benchmark local models per task. It is very thorough and has always returned good setups. The only thing you have to do is make sure you point at the model card. It’s always incredulous that models exist after its training cutoff.

K3 does an ok job of setting up, but its config searching isn’t nearly as thorough and its will confidently tell you it’s found the best setup when it’s only turned a few knobs. It’s also not a good evaluator of its own output. It rates its work too highly and seems kind of defensive when benchmarking. Still a good check because it does find stuff, but open a fresh session and don’t tell it where the results came from.

What K3 does do more than any other model I’ve found is investigate folder structures. If I want to benchmark it and other models I have to move the testing methodology and any reference to other results out of the project folder bc Kimi is like an ls bloodhound. It will find them.

Post reply on HN