Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

21–30 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#21

Post by wongarsu »

alightsoul wrote: post #7

how is it possible to distill fable only a month after its release? maybe they are confusing opus with fable.

If by "distill" they mean "used it for fine-tuning" then they might have used it in the final stages of fine-tuning of Kimi K3. I image they might have already been using Opus, and when Fable became available it was easy to switch over to it

It would have been a tiny part of the overall training, given the timeline

Re: “We have information that Moonshot distilled Fable for the development of K3”

#22

Post by sosodev »

alightsoul wrote: post #7

how is it possible to distill fable only a month after its release? maybe they are confusing opus with fable.

A month seems plenty long enough. They're not rebuilding the entire model from scratch. It's just getting Fable to act as a teacher model for some of the final reinforcement learning on the base that Kimi already had.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#23

Post by supriyo-biswas »

dang wrote: post #3
treetalker wrote: post #2

rules for thee but not for me

Plenty of HN readers feel this way and it's a good point, but it has also become an entirely cliché response which pops up like mushrooms anytime "distillation" appears. That means it's against the site guidelines, which ask: " Eschew flamebait. Avoid generic tangents. Omit internet tropes. " - https://news.ycombinator.com/newsguidelines.html I don't mean to pick on you personally! It's just that reflexive responses…

I think then we should ban these sorts of posts about the allegation of distillation, since being able to post the story but then warning accounts with comments about the hypocrisy, is not the correct way to go about it.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#24

Post by unethical_ban »

treetalker wrote: post #2

rules for thee but not for me

Let me put it in an HN-acceptable format:

Given the disregard for intellectual property rights the AI labs had in creating the technology, many people feel no sympathy for second-order AI labs using similar techniques to build technology off the US frontier labs.

I think fighting distillation will always be cat-and-mouse, and that it's more of a concern for the stockholders and perhaps an iota of national security. It can't be stopped entirely; the "problem" will always be there.

I'm much more concerned about asymmetry of power between citizens and their governments with omnipresent surveillance and analysis being done on everyone living their lives. Societies throughout history have taken as a given their power to overthrow malicious governments when things hit a breaking point, and I am scared that this technology will lock societies into a state of total subordination for eternity.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#25

Post by kouteiheika »

Assuming they did then they surely paid for them, which makes it "not stealing". Am I also "stealing proprietary U.S. technology" by harvesting my Claude chats from my `.claude` directory and training a bunch of models on them?

That said, I doubt the "they distilled Fable" is the reason why K3 is as good as it is, considering the timelines involved, and that Anthropic hides thinking traces, and their overly aggressive "safety" filters.

This constant FUD spread by Anthropic is so tiring.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#26

Post by cute_boi »

throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

Even if they distilled this crappy politician should have no issue. Anthropic pirated whole ebook collection and millions of github repo with gpl license.

We should do more distillation and figure out how to create faster leaner and better models.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#27

Post by Catloafdev »

I wonder how they detect this kind of thing. Seems like this is going to be a perpetual issue until it stops being worth doing.

Side note, didn't they stop releasing real thinking tokens for Fable? Or is it still part of some subs or API usage?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#28

Post by sosodev »

throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

Distillation is a very vague term. It can mean anything from training exclusively on a model's output to using it for a very small portion of the training. In this case it is almost certainly towards the very small portion side of the spectrum.
Post reply on HN