Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

231–240 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#231

Post by xcf_seetan »

make3 wrote: post #64
madduci wrote: post #55

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime

It's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an important point, and it's likely not the case. It's also about the larger companies explaining why they can't be as efficient, of course they can't, they're not just ripping the outputs of another model that someone else invested billions to train.

> they're not just ripping the outputs of another model that someone else invested billions to train.

If they payed for inference, doesn't they own the output? So if I pay for a model to generate code, isn't that code mine to do with it whatever I want? Just curious.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#232

Post by bakugo »

Fun fact about K3's distillation:

As of a couple months ago, when using Claude to write adult content through the API, sometimes it will silently inject a system prompt giving the model a bunch of guidelines on exactly what kind of adult content it's allowed to write, steering it away from anything "questionable" ("Claude will not write etc etc").

Moonshot distilled Claude so hard recently, they actually ended up distilling this prompt injection, too. Using K3 to write adult content results in it randomly hallucinating the injected Claude prompt during thinking, and it will quote parts of that prompt, complete with the name "Claude".

Not that I think distillation is a bad thing, just thought this was funny.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#233

Post by fc417fc802 »

robotpepi wrote: post #194
linkregister wrote: post #167

Earlier quoted context omitted.

It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other le…

Chatgpt routinely cites and uses papers I don't have access to because they're behind a paywall. I don't think OpenAI is paying for all that copyright. That's in my opinion way more serious.

Yes the fact that the scientific literature - created largely on the back of the tax payer - isn't open to all free of charge by force of law is a travesty. A cartel should not get to charge for access to the bulk of human knowledge. That is indeed a far more important issue than whether or not Moonshot violated the Anthropic ToS, possibly committing mass fraud in the course of doing so.

I mean honestly if they did that why should I care? I'm happy to see copyright violated in a manner that leads to the creation of new technology. IP law exists strictly for the benefit of society and by all appearances AI is an incredibly powerful tool.

Also while I'm at it libgen is a gift to humanity. Information wants to be free. Spreading and preserving knowledge is generally one of the most wholesome activities anyone can undertake as far as I'm concerned.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#235

Post by himata4113 »

atleastoptimal wrote: post #215
himata4113 wrote: post #200

Earlier quoted context omitted.

Never claimed that it is irrelevant. And kimi k3 is on the same level and sometimes outperforms fable 5 - that cannot be explained by distillation. The reason why gap is not closed is simply the fact that fable was trained months ago so in theory the frontier labs are still 1 (small) step ahead. Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competiti…

Closed-source models have to deal with the current frontier being heavily regulated. Fable, at its old level, was "too good" to be released and they had to add an additional safety layer to sanitize the outputs. Lowering the quality of the models so they are safer and more steerable has been something all the closed-source models have been doing for a while, a requirement that many open source models don't need to de…

I recommend reading some of their research it's honestly astonishing how intelligent some of their solutions are.

Kimi specifically relies heavily on reasoning traces which is largely due to their training strategy and will perform poorly when thrown into a conversation from another model. Another fun advancement is that they simply ctrl+c ctrl+v'd attention which means that the model can steer where to look in the context window without ever producing an output token increasing token efficiency and attention accuracy as a side effect you end up with weaker prompt adherence.

None of these 'issues' manifest in US models which proves that kimi has diverged and is achieving these capabilities seperately from the architecture that US labs rely on.

I would agree with you during the Deepseek R1 era, but US labs were heavily inspired by open research at that point as well so I wouldn't give them too much credit.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#238

Post by FpUser »

smeeth wrote: post #222
FpUser wrote: post #201

Earlier quoted context omitted.

>"A lack of prosecution does not make something legal" Plainly who gives a flying fuck. The US can claim whatever rules they want and so can China or any other country. On international level all those rules are artificial constructs unless they can be enforced. China can just say for example that they do not recognize copyrights /patents / whatever so it is "legal" for them.

This is illegal in China too, there's just an enforcement asymmetry. I understand what you're saying is de facto true, I'm just taking issue with people saying either 1) its not illegal (it is) 2) it shouldn't be illegal because Anthropic stole training data (thats not how the law works)

>"1) its not illegal (it is)"

I am a practical man. From what I see laws are mostly for common folks and often do not even serve real justice. The higher one goes and the amount of money / power involved the more the laws bend and on international level the only law that matters is the size of one's club and willingness to use it. And when the country with supposedly biggest one starts crying I find it laughable.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#239

Post by gensym »

chasd00 wrote: post #226

if there are no consequences then who cares? You're not going to take Chinese companies to court and stealing IP is nothing new either. It's going to take some sort of policy change at the federal government level to do anything but they haven't done much up to this point. Maybe AI is important enough to actually get some kind of policy change, sucks for everyone else who have had their IP stolen with no consequences…

I fear they are laying the groundwork to ban US citizens from using Chinese models, so that they can make sure that Brockman gets his money's worth.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#240

Post by brap »

Anyone surprised by this is incredibly naive.

By all means use whatever works for you, I’m not even going to try to make an argument on ethics (and honestly I’m not even sure where I stand, given the behavior of American AI companies).

But I just cringe every time I see people acting like any of this is done in good faith.

Open source coming out of China is a state-sponsored criminal enterprise, built only for the benefit of the Chinese regime, one of the worst to exist in human history.

Post reply on HN