Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

171–180 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#171

Post by noja »

linkregister wrote: post #167
himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other le…

Isn’t that the same argument they are making for replacing human labour?

Circumventing costs.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#172

Post by stephbook »

I don't know what purpose these "they copied us" crying is ever going to achieve. Europeans stole Chinese silk worms. US stole European books, looms and rocket scientists. Who cares? Be grateful you've got people inventing stuff worth copying.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#174

Post by syrrim »

SwellJoe wrote: post #157
pandinus wrote: post #102

As with many others among these threads I don't see how the timing works out for K3 to have trained on distilled Fable usage. There should be at least a tacit academic acknowledgment of Kimi's own design efforts. Distillation itself, however, is still clearly valuable - else competitors wouldn't pay so much to their rival on distillation campaigns or try to circumvent anti-distillation defenses. As for the morality o…

"It is already understood that you own the output." I don't think it's settled that anybody owns the output. There seems to be some question whether LLM output can be copyrighted (and there should be). I'd rather it weren't possible, actually. I think it's better for humanity if we acknowledge that what was legitimately ingested into these models is our collective commons (and what was illegitimately ingested into th…

There's no question about it. Llm output is not copyrightable. Which is moot anyways, because training on copyrighted data is completely legal.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#175

Post by mattertoast »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

It does matter in that these LLM companies need to be run into the ground, and every embarrassing clod working for them run out of town.

It's showing that 'distillation' is a viable way to reclaim all of what they stole and hoard, and with enough luck their debts will come due in time for them to feel it.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#176

Post by mrandish »

gruez wrote: post #44
bradfa wrote: post #37

I can understand that the AI labs might care about other labs distilling their models as it can eat into their competitive advantage, but do consumers care at all? Aren't consumers benefiting from this practice by getting better cheaper models as a result?

They're probably going for the national security/domestic manufacturing angle. > Aren't consumers benefiting from this practice by getting better cheaper models as a result? Aren't consumers benefiting from cheap chinese batteries, EVs, and drones?

> cheap chinese batteries, EVs, and drones?

The banning or effective banning through tariffs of products like EVs is a pretty dumb economic strategy that rarely works out in the long-run.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#177

Post by onraglanroad »

unethical_ban wrote: post #24
treetalker wrote: post #2

rules for thee but not for me

Let me put it in an HN-acceptable format: Given the disregard for intellectual property rights the AI labs had in creating the technology, many people feel no sympathy for second-order AI labs using similar techniques to build technology off the US frontier labs. I think fighting distillation will always be cat-and-mouse, and that it's more of a concern for the stockholders and perhaps an iota of national security. I…

That's a better comment but

> Societies throughout history have taken as a given their power to overthrow malicious governments when things hit a breaking point,

simply isn't true. People throughout history have simply accepted that society is the way it is and sometimes used whatever means they could to get to the top.

Revolutions have been very rare and usually ended up with the revolutionaries simply taking the place of the previous rulers. "Meet the new boss..."

Re: “We have information that Moonshot distilled Fable for the development of K3”

#178

Post by kevinqi »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#179

Post by mNovak »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

> The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens

Note that Chinese companies are free to rent from GB300 clouds internationally. There are large datacenter hubs in Singapore and Malaysia serving chinese and other customers.

Though there is also reported [1] significant smuggling of Nvidia chips into China as well.

[1] https://epoch.ai/publications/chip-smuggling

Re: “We have information that Moonshot distilled Fable for the development of K3”

#180

Post by Dylan16807 »

smeeth wrote: post #163
zaptheimpaler wrote: post #155

Earlier quoted context omitted.

All of the models stole the entirety of written knowledge on the internet to train. They are being sued for the few cases where we have some proof of what they did because of some whistleblowers, all the rest will just go unpunished. They breached Github TOS, robot.txt's, copyright, patents every form of IP protection under the sun from a billion sources. It's just ridiculous for the thieves to cry about someone else…

Do you care about the law or not? I think theft is bad everywhere, not just when Anthropic does it.

For me, when it specifically comes to copying, I don't think it's bad to copy a copier. (And by that I mean Anthropic has no valid complaints against Moonshot. Any valid complaints from anyone in the original corpus are valid against both of them now.)

In this way, it is different from literal theft. Stealing money/objects from a thief and keeping them is not justified.

Post reply on HN