Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

wafer.ai

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#61

Post by anon373839 »

croes wrote: post #51
Almondsetat wrote: post #48

Earlier quoted context omitted.

Sorry, where exactly is "data" in "source"? I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different

For LLMs the training data is the source of their weights. You can‘t reproduce the LLM without the same data

That’s not really true. The models have source code defining their architecture and it is open-source.

People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code.

So they’re a very valuable asset that complements the code, but they are not the code. You could use randomly initialized weights and the software will work - it will output tokens. They just won’t have useful patterns.

I don’t think OSS definitions have ever required that assets have their source included. For example, artwork is very important to a game, but nobody thinks a game is not open source if it doesn’t come with sketches and a copy of Adobe Illustrator to recreate the artwork from scratch.

Edit: The distinction I would draw between models like Deepseek and models like OLMo is whether they are open science. With a model like OLMo, they have published everything you need to replicate the training experiment. Whereas Deepseek does share a lot of knowledge, but keeps a lot proprietary too.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#62

Post by anon373839 »

halJordan wrote: post #55
WithinReason wrote: post #46

Earlier quoted context omitted.

The weights are the modifiable representation. You modify them with gradient updates.

I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?

The antenna firmware is software. Model weights are essentially learned settings, and they are not software.

Note also that software is copyrightable and model weights aren’t (in the US anyway).

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#63

Post by pocketarc »

anon373839 wrote: post #61
croes wrote: post #51

Earlier quoted context omitted.

For LLMs the training data is the source of their weights. You can‘t reproduce the LLM without the same data

That’s not really true. The models have source code defining their architecture and it is open-source. People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code. So they’re a very valuable asset that complements the code, bu…

Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#64

Post by inigyou »

WithinReason wrote: post #46
HPsquared wrote: post #43

Earlier quoted context omitted.

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights. The weights are literally a binary blob.

The weights are the modifiable representation. You modify them with gradient updates.

You can hand-patch a binary, too.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#65

Post by anon373839 »

pocketarc wrote: post #63
anon373839 wrote: post #61

Earlier quoted context omitted.

That’s not really true. The models have source code defining their architecture and it is open-source. People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code. So they’re a very valuable asset that complements the code, bu…

Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.

I think this misses the point I was trying to make. Both an open source game and an open weights LLM include the necessary assets. But people seem to be saying, “you need everything that was used to create the assets”. For game art, that would include sketches, prior art used as inspiration, project files that aren’t used by the game engine but were used during the design process, and so on - if you apply the same standard. But I don’t think anyone does.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#66

Post by BoxOfRain »

pocketarc wrote: post #63
anon373839 wrote: post #61

Earlier quoted context omitted.

That’s not really true. The models have source code defining their architecture and it is open-source. People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code. So they’re a very valuable asset that complements the code, bu…

Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.

Doom pioneered this model, the Doom engine was open-sourced while the assets remained proprietary. This is where projects like Freedoom exist, to create an entirely free software equivalent.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#67

Post by andy99 »

charcircuit wrote: post #24
teruakohatu wrote: post #13

Earlier quoted context omitted.

> There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project. Open source means open source code. Open weight means a binary file dump, not unlike an exe file. There is nothing open source about it. Its like having a closed source text editor that censors certain words, and an open source text editor that censors certain words. The latter…

I think you are failing to see how the weights are the preferred form of working with a model. It's like if I shared a png with others to use as a meme template. Even if I don't share the source code to photoshop other people can use that "binary file dump" to make derivative images of their own with it.

I feel like the arguments being made about sharing the training data are from people who don’t have any experience training ML models and have never worked with weights. The biggest reason I have seen that people want the data is to complain about some aspect of it (copyright, bias, etc)

I would be very interested to hear someone explain what freedom it is they need the training data to enjoy.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#68

Post by croemer »

anon373839 wrote: post #62
halJordan wrote: post #55

Earlier quoted context omitted.

I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?

The antenna firmware is software. Model weights are essentially learned settings, and they are not software. Note also that software is copyrightable and model weights aren’t (in the US anyway).

Not sure why it matters whether weights are software or not.

One can equally argue that firmware is just settings for the CPU.

If weights aren't copyrightable then doesn't this argue against the idea of them being able to have the quality of open-sourceness? Meanwhile the training code could be copyrightable.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#69

Post by gpm »

pocketarc wrote: post #63
anon373839 wrote: post #61

Earlier quoted context omitted.

That’s not really true. The models have source code defining their architecture and it is open-source. People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code. So they’re a very valuable asset that complements the code, bu…

Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.

People refer to games with proprietary assets as open source all the time. The source code is open - the assets are not. For one example: https://news.ycombinator.com/item?id=28201265 / https://github.com/beyond-all-reason/Beyond-All-Reason/blob/...

Source as a word has a meaning that doesn't include non-source things.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#70

Post by gpugreg »

halJordan wrote: post #56
gpugreg wrote: post #33

Earlier quoted context omitted.

> Some of us might be rich I sure wish I had a few 100M of disposable income to train a frontier model. > or in the future it could be useful when training is cheaper. I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

You say that but for like 2 years there was a guy on huggingface releasing quants as TheBloke. No identity no nothing except he likely had a grant or a university job that let him do it. Quants aren't the same as training but it was beyond 99% of people at the time (as you say this is) Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.

I did not say that it is impossible. I just think that we need architectural improvements, or maybe even a fundamentally different approach to get something like Kimi K3 for cheap. The point I was trying to make was that we shouldn't just laze about and hope that hardware improvements will get us there.

(Also, I know Tom Jobbins (TheBloke), and have personally contributed to increase the adoption of GGUF, e.g. in the transformers and ktransformers libraries, so I find the personal dig quite amusing.)

Post reply on HN