Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

301–310 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#301

Post by linkregister »

Commenters are overlooking the significance of this information and posting emotional reactions based on perceptions of fairness or feelings of schadenfreude.

The economic viability of Anthropic and OpenAI rely on their being able to charge more for model access than their R&D and inference costs. If the market price for SOTA model access drops below that level, then these businesses will have to decide whether to continue to lose money or to reduce spending on R&D.

Moonshot's papers [1] claim that their training load was primarily from synthetic data and model self-teaching rather than RLHF and therefore keep their costs low. If Moonshot genuinely does not rely on human-led training, they will surpass US closed-source model providers. The United States government considers US supremacy in "AI" as a national security consideration.

This announcement is noteworthy because it implies that Moonshot's success is in fact due to distillation. It's in the interest of US frontier labs to place barriers to this if they find themselves in the position of subsidizing rival labs' research.

1. Kimi K2, https://arxiv.org/html/2507.20534v1

Re: “We have information that Moonshot distilled Fable for the development of K3”

#302

Post by asadotzler »

smeeth wrote: post #206
Dylan16807 wrote: post #180

Earlier quoted context omitted.

For me, when it specifically comes to copying, I don't think it's bad to copy a copier. (And by that I mean Anthropic has no valid complaints against Moonshot. Any valid complaints from anyone in the original corpus are valid against both of them now.) In this way, it is different from literal theft. Stealing money/objects from a thief and keeping them is not justified.

It's a little different in this case, since 1) not all the data Ant used was stolen and 2) they did contribute significantly to the value of the stolen good. An analogy might be a baker stole 20% of the flour used to bake their special bread, which was then stolen. Both thefts are obviously wrong and bad.

The baker stole 100% of the flour to make the bread. He also stole the water and the salt and the yeast and the heat for his oven. What he didn't steal was the time he put into crafting a recipe for bread and the time he sat around waiting for the oven to bake it. Now, is that loaf stolen property? Hard to say. But the baker is undoubtedly a thief. He should be tried and forced to pay restitution out of his ill-gotten profits for sure. If we can't do that, the next step is pitchforks and guillotines.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#303

Post by evanelias »

ffsm8 wrote: post #214
andyfilms1 wrote: post #188

Earlier quoted context omitted.

Oh, so mass theft is okay as long as American companies are doing it

copyright infringement is not theft, even if right holders often claim it is. part of the definition of theft is that the original owner is deprived of it, which does not apply to copyright infringement. You can only argue with damages from the perspective of potential profits, still not theft though. https://en.wikipedia.org/wiki/Theft

If you steal an unpopular product from a store, the damage is also only to "potential profits", so how does that differ? It's entirely possible no one would have purchased the product and it would have eventually been discarded/destroyed.

Or with services, if a barber cuts your hair and then you run away without paying them, do you not consider that theft, even though there's no change in ownership occurring?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#304

Post by SubiculumCode »

trollbridge wrote: post #277
SubiculumCode wrote: post #227

Earlier quoted context omitted.

Moreover, reading a copyrighted book and learning from it is not theft.

Great! Neither is distillation then.

Never said it was. Still, understanding to what extent the ability of Chinese labs to keep up to western models with much less compute needs to be understood.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#305

Post by linkregister »

andyfilms1 wrote: post #188
linkregister wrote: post #167

Earlier quoted context omitted.

It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other le…

Oh, so mass theft is okay as long as American companies are doing it

Reread my comment and look for a value judgement on my part. The final sentence is probably a good clue as to my opinion.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#307

Post by amanaplanacanal »

catigula wrote: post #105
Matl wrote: post #98

Earlier quoted context omitted.

> So what is the issue here? The issue seems to be the US only likes competition when it is winning. Markets in Asia are meant for cheap labor and resources, they're not meant to actually compete. /s

[flagged]

What IP is being stolen here? So far, the courts have ruled that anything generated by an LLM is not copyrightable.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#308

Post by linkregister »

robotpepi wrote: post #194
linkregister wrote: post #167

Earlier quoted context omitted.

It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other le…

Chatgpt routinely cites and uses papers I don't have access to because they're behind a paywall. I don't think OpenAI is paying for all that copyright. That's in my opinion way more serious.

Your statement is orthogonal to my comment. Why reiterate the schadenfreude / fairness comment already stated several dozen times in this thread?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#309

Post by il »

remus wrote: post #265
jaggederest wrote: post #142

Earlier quoted context omitted.

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…

Probably not as much effort as writing books and creating art the models were trained on.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#310

Post by SubiculumCode »

bigfishrunning wrote: post #279
SubiculumCode wrote: post #227

Earlier quoted context omitted.

Moreover, reading a copyrighted book and learning from it is not theft.

Generating a set of weights is not learning.

That is a strong statement. I guess you are telling Machine Learning to go fuck itself.
Post reply on HN