Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

wafer.ai

Post reply on HN
Print view View original

81–90 of 119 posts

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#81

Post by SV_BubbleTime »

fooker wrote: post #80
kelnos wrote: post #10

I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model. They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

This trope was valid maybe in 2022. Model training now is not a straight forwards process of input data -> run tools -> get model. There's a whole lot of alchemy going on. We don't quite understand what works and what doesn't. Think of it like painting with water color and having to improvise very often. The only advantage over water color is that we can revert to a working state.

And since none of the source is provided to make the thing, it’s not open source.

Really, it’s no more open source than a free calculator. It’s free, and you can use it to no or great effect. But, you sure as hell can’t make one.

So calling it open source is without question, wrong.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#82

Post by SV_BubbleTime »

gpm wrote: post #69
pocketarc wrote: post #63

Earlier quoted context omitted.

Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.

People refer to games with proprietary assets as open source all the time. The source code is open - the assets are not. For one example: https://news.ycombinator.com/item?id=28201265 / https://github.com/beyond-all-reason/Beyond-All-Reason/blob/... Source as a word has a meaning that doesn't include non-source things.

That’s an open source engine, we already have a word to define what that is.

And we have Open Weights to define what those are.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#83

Post by gpm »

SV_BubbleTime wrote: post #82
gpm wrote: post #69

Earlier quoted context omitted.

People refer to games with proprietary assets as open source all the time. The source code is open - the assets are not. For one example: https://news.ycombinator.com/item?id=28201265 / https://github.com/beyond-all-reason/Beyond-All-Reason/blob/... Source as a word has a meaning that doesn't include non-source things.

That’s an open source engine , we already have a word to define what that is. And we have Open Weights to define what those are.

No, an engine is only the code that is shared between games. For instance "urban terror" by frozen sands is a closed source game on top of a (now) open source engine (the Quake3 engine). Quake3 is a (now) open source game (with closed assets) on top of its own open source engine. Quake and it's derivatives are a particularly nice example of the distinction between a game and its engine because the engine includes an interpreter for the game code, leaving a very clear boundary. Meanwhile many games end up intermingling their engine and game code (e.g. see the Eve Online devs write about this - another closed source game on top of a now open source engine).

BAR is an open source game. All the game source is open.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#84

Post by HPsquared »

ungovernableCat wrote: post #79
HPsquared wrote: post #54

Earlier quoted context omitted.

They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.

If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?

They don't even say what the ingredients are.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#86

Post by fooker »

SV_BubbleTime wrote: post #81
fooker wrote: post #80

Earlier quoted context omitted.

This trope was valid maybe in 2022. Model training now is not a straight forwards process of input data -> run tools -> get model. There's a whole lot of alchemy going on. We don't quite understand what works and what doesn't. Think of it like painting with water color and having to improvise very often. The only advantage over water color is that we can revert to a working state.

And since none of the source is provided to make the thing, it’s not open source. Really, it’s no more open source than a free calculator. It’s free, and you can use it to no or great effect. But, you sure as hell can’t make one. So calling it open source is without question, wrong.

There's no program you run to 'make the thing'.

It's mostly ad hoc scripts and some pretty horrible hacks being run by a hundred engineers trying to improve a thousand different things at once with a hundred thousand GPUs.

I'm sure once we understand the tech better, the training process will look like running a program.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#87

Post by ungovernableCat »

HPsquared wrote: post #84
ungovernableCat wrote: post #79

Earlier quoted context omitted.

If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?

They don't even say what the ingredients are.

Yeah I understand that. I mean just the methods they use to acquire the training data has already had a shit ton of drama, there's no way the western labs are ever going to be public about the ingredients.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#88

Post by ordersofmag »

HPsquared wrote: post #44
mlazos wrote: post #38

Earlier quoted context omitted.

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

Model weights are literally a compressed blob of binary data. The end result of a compilation process.

So is a zip file of source code. There is no 'compilation process'. Model training isn't a fixed end point. You start with random weights. You train. The weights get better. You do this iteratively. At some point you say 'good enough' and release. People doing further training would start with those weights and further iterate. Demanding the original training data and training process would be equivalent to demanding a completely replay-able sequence of editing steps starting with a blank text file and allowing you to replay ever edit that led the original author to the released source code.

Now there are reasons you might want to know about the training data when you wouldn't care about the authoring process used by a traditional open-source process. And these get at the reason LLM's are different than traditional software and so maybe our existing definitions of what 'open-source' means aren't a good match for LLM's. Of course there is software associated with LLM's (beyond the weight) -- defining the structure of the particular neural net those weights fit into. In every open-weights model that I'm aware of that software is open source (though trivial).

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#89

Post by SV_BubbleTime »

gpm wrote: post #83
SV_BubbleTime wrote: post #82

Earlier quoted context omitted.

That’s an open source engine , we already have a word to define what that is. And we have Open Weights to define what those are.

No, an engine is only the code that is shared between games. For instance "urban terror" by frozen sands is a closed source game on top of a (now) open source engine (the Quake3 engine). Quake3 is a (now) open source game (with closed assets) on top of its own open source engine. Quake and it's derivatives are a particularly nice example of the distinction between a game and its engine because the engine includes an…

Bro, your link literally says in the read me it is an engine.

Re: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300

#90

Post by gpm »

SV_BubbleTime wrote: post #89
gpm wrote: post #83

Earlier quoted context omitted.

No, an engine is only the code that is shared between games. For instance "urban terror" by frozen sands is a closed source game on top of a (now) open source engine (the Quake3 engine). Quake3 is a (now) open source game (with closed assets) on top of its own open source engine. Quake and it's derivatives are a particularly nice example of the distinction between a game and its engine because the engine includes an…

Bro, your link literally says in the read me it is an engine.

No? It says

> Open source RTS game built on top of the Recoil RTS Engine

BAR is a game, Recoil (a different thing - not the repository I linked) is an engine it is built on top of. Recoil is - incidentally - a fork of spring, an engine many open source RTSes are built on.

Post reply on HN