Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

wafer.ai

Post reply on HN
Print view View original

51–60 of 119 posts

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#51

Post by croes »

Almondsetat wrote: post #48
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

Sorry, where exactly is "data" in "source"? I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different

For LLMs the training data is the source of their weights.

You can‘t reproduce the LLM without the same data

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#53

Post by croes »

swiftcoder wrote: post #27
croemer wrote: post #26

Earlier quoted context omitted.

Yes, obviously.

Even though none of us could actually afford to train it?

Yet, I couldn’t afford a PC to run the original Crysis when it came out, now the hardware isn’t an issue anymore

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#54

Post by HPsquared »

WithinReason wrote: post #46
HPsquared wrote: post #43

Earlier quoted context omitted.

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights. The weights are literally a binary blob.

The weights are the modifiable representation. You modify them with gradient updates.

They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#55

Post by halJordan »

WithinReason wrote: post #46
HPsquared wrote: post #43

Earlier quoted context omitted.

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights. The weights are literally a binary blob.

The weights are the modifiable representation. You modify them with gradient updates.

I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#56

Post by halJordan »

gpugreg wrote: post #33
croemer wrote: post #30

Earlier quoted context omitted.

Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper. Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.

> Some of us might be rich I sure wish I had a few 100M of disposable income to train a frontier model. > or in the future it could be useful when training is cheaper. I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

You say that but for like 2 years there was a guy on huggingface releasing quants as TheBloke. No identity no nothing except he likely had a grant or a university job that let him do it. Quants aren't the same as training but it was beyond 99% of people at the time (as you say this is)

Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#57

Post by applfanboysbgon »

ciupicri wrote: post #25
Alpha3031 wrote: post #17

Earlier quoted context omitted.

Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source. I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point…

You can also edit/patch binary executables, but no one says about that software that is open source.

Incidentally, I do. I fucking hate the open source maximalists like Stallman who suggest that not having source code is equivalent to not having control over what runs on your computer. But you always[1] have control over what runs on your computer. Reverse engineer it if you care so much! Every form of DRM is inevitably defeated, because you do have control over program execution, regardless of access to the source code. Stallman's argument seems more lazy and entitled than principled.

[1] ...setting aside hardware attestation bullshit, which does need to die in a fire.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#58

Post by solarkraft »

WithinReason wrote: post #20
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

If they open sourced the training data and code but you had to train the network yourself, would that be open source?

Yes

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#60

Post by girvo »

WithinReason wrote: post #20
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

If they open sourced the training data and code but you had to train the network yourself, would that be open source?

Which some companies have done! Nvidia, I believe, among others.
Post reply on HN