Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

141–150 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#141

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

It also doesn't matter for a simpler, and much grander reason.

All LLMs are trained on the corpus of humanity's knowledge, the legacy of everyone who's ever lived and our civilization as a whole.

Anything that prevents or circumvents the accumulation or gatekeeping of this knowledge and puts it in the hands of more people (that are not AI company shareholders) is a good thing. Whether that is done by open sourcing the model weights, the training set, or by making the output better and cheaper, it is all fair game and is, as another poster mentioned, inevitable in the long run.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#142

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#143
post #43

Didn't they just pay a fine for stealing all those books?

$1.5 billion fine for downloading 7 million books from LibGen and other pirate torrents. That's also the case where the judge ruled that training AI models on books could qualify as fair use, but storing millions of pirated works in a central internal library without licensing constituted copyright infringement. It will be interesting to see if courts consider training on data distilled from a model fair use. Assumin…

Well I mean it still worked out for them because they wouldn’t have had the 1.5 billion to license before doing the training and the company exploding into a trillion dollar company?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#144

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

Uh, what?

> Distillation is not illegal by every definition of the word

Note that Anthropic (and USG) alleges [0] not only that Kimi was distilled, but that they actively circumvented measures intended to stop distillation. There are multiple ways that's illegal, including:

- Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done.

- Economic espionage: 18 U.S.C. §1831 criminalizes obtaining a trade secret through theft, fraud, or deception while intending that it will benefit a foreign entity.

- Trade-secret misappropriation: if Anthropic could argue industrial-scale querying reconstructed proprietary aspects of Fable (like by showing it produces similar outputs, as others have done) then it's illegal under 18 U.S.C. §1832.

- California computer-access statute §502 bars knowingly accessing a computer system and, without permission, taking, copying, or using its data.

- Computer Fraud and Abuse Act protects against the case where restrictions against an activity are circumvented (like Kimi is alleged to have done).

> There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them.

A lack of prosecution does not make something legal. There is also the scale/commercialization thing, which isn't an issue with random tiny HF datasets/models. Remember: Kimi also sells K3 inference.

> kimi architecture is vastly different than that of fable

How do you know that? Do you work for Anthropic? Also, this has nothing to do with architecture, we are talking about data.

> US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable.

Cool. The difference is that one of those things is legal (because they chose to open-source) and one of those things is illegal theft of trade secrets (because it was stolen).

> Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.

1) this has nothing to do with other labs, just Moonshot (and Z.ai, MiniMax, DS)

2) slandering or not it happens to be completely true, so, there's that

[0] https://www.anthropic.com/news/detecting-and-preventing-dist...

Re: “We have information that Moonshot distilled Fable for the development of K3”

#145
post #98

Earlier quoted context omitted.

> So what is the issue here? The issue seems to be the US only likes competition when it is winning. Markets in Asia are meant for cheap labor and resources, they're not meant to actually compete. /s

[flagged]

Oh, ok. How would you describe how the "frontier us companies" acquired the data used to form their models?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#146
post #55

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime

Distilling is still fair

I generally agree, in the same sense that it's "fair" for the US and China to spy on each other. It's not a moral outrage, but it is something that the targets can and should try to prevent.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#147
post #55

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime

I'm by no means taking the side of the AI companies, but it's possible that Anthropic "added value" to the data they harvested. Stealing that does seem kind of uncool. Regardless, it was always inevitable—will continue to happen.

Valuation is hard to perform when it's deep inside a black box. Ther "API" may be easier to evaluate. The problem with this angle is that Moonshot is actually producing _better_ value from Anthropic's blackbox.

Technically, providing better value from your competitor's private holdings could be theft (of trade secrets), but might it also be fair use? "Schrodinger's IP" be damned.

I don't think the 1.5B settlement has resolved this. The 2 cases need to be merged!

Re: “We have information that Moonshot distilled Fable for the development of K3”

#148

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

[dead]

Re: “We have information that Moonshot distilled Fable for the development of K3”

#149
post #137
post #90

Earlier quoted context omitted.

> on the same level like Anthropic scraped copyright protected material for their training. I see no problem with distillation, on the other hand the complete dismissal of copyright by AI labs is pretty bad, I don’t think we should put them at the same level

> on the other hand the complete dismissal of copyright by AI labs Courts keep ruling over and over that an LLM trained on copyrighted works qualifies as a transformative work and is therefore fair use. They don't have to dismiss copyright law, this has always been allowed. The only thing they get in trouble for is pirating the works to get their hands on them.

"Keep ruling over and over" is way too strong. There have maybe been two rulings, nothing nationally binding, and most of the litigation is still ongoing. In particular, last I checked OpenAI and Microsoft are still badly threatened by the NYT lawsuit: https://law.justia.com/cases/federal/district-courts/new-yor... https://www.cnet.com/tech/services-and-software/publishers-o...

This will have to wait for the Supreme Court. OpenAI and Microsoft 100% deserve to lose, even without OpenAI allegedly hiding evidence.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#150
post #98

Earlier quoted context omitted.

> So what is the issue here? The issue seems to be the US only likes competition when it is winning. Markets in Asia are meant for cheap labor and resources, they're not meant to actually compete. /s

[flagged]

Stealing IP is how American industry got started. Goose: gander, pot: kettle.

It is what built and sustains the movie and music industries. See: work for hire and 100+year copyright length

The tech industry: see: copyright and patent assignment from discoverer to corporation.

I know that corporations forcing me to assign patents and copyright to them was an incentive to take published works from "software practice and experience" and other technical journals, use them as the core of my work, and disclose that source to the company I worked for. Didn't stop them from applying for patents, however.

I think the discussion of copyright needs more refinement. We need to separate the discoverer's need for acknowledgment of development effort from the rent-seeking core of copyright.

Post reply on HN