Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

521–530 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#521

Post by HNisCIS »

nylonstrung wrote: post #298
throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

If distillation truly is the cheat code they act like it is, then all the US and EU AI labs have no excuse for not having Fable-level models already

I took a picture of a jpeg and compressed it as a jpeg for extra jpeg

Re: “We have information that Moonshot distilled Fable for the development of K3”

#522

Post by bigyabai »

some_random wrote: post #424
matheusmoreira wrote: post #420

Earlier quoted context omitted.

> The United States government considers US supremacy in "AI" as a national security consideration. And we foreigners consider US supremacy in AI to be an existential threat. Your "national security" is directly harmful to us. I never thought I'd say this but the chinese are starting to look like a beacon of hope for the rest of us.

If the Chinese look like a beacon of hope you then you really should be looking closer.

Explain it, then. Don't just wimp-out with trite allusions to nothingness. Discredit them.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#523

Post by sailingparrot »

dudeinhawaii wrote: post #511

I think most of the comments in this thread are missing the point. It's not about whether it's legal/ethical/etc. It's about the narrative that "Chinese models are at Fable level". The truth (if correct) is the China continues to copy, and the proprietary US Models continue to lead the state of the art. There is no K4 without Fable 6, GPT-6. That, matters.

> There is no K4 without Fable 6, GPT-6. That, matters.

That's simply not true though. Chinese labs very clearly have the entire stack developed and working. Using traces from claude allows them to shorten their training time by some amount, that's it.

Remove Fable 6 and you still have K4 eventually, just 2 months later at best.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#524

Post by thewebguyd »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

> Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.

They claim it because Anthropic are planning to push for protectionism. They just doubled their political spending to $40 million for the midterms to "push for AI regulation" Gee, I wonder what it is they are lobbying for. Certainly won't be OFAC sanctions right? ICTS import controls?

US GOV, under lobbying pressure from Anthropic and OpenAI are going to go full protectionism and restrict Chinese models, I'd almost be willing to bet money on it. They can't really enforce for individuals, but they can definitely tell US based hpyerscalers they can't host them, make it illegal to host the weights, and government procurement restrictions.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#525

Post by moralestapia »

kami23 wrote: post #468
moralestapia wrote: post #372

Earlier quoted context omitted.

How does that connect with @throwa356262's argument?

That they should be able to find distillation 'attacks' if they had enough observability.

That's not @throwa356262's argument.

@throwa356262 argument is that it is infeasible to distill and release a new frontier model in two weeks.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#526

Post by tristanj »

throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

[dead]

Re: “We have information that Moonshot distilled Fable for the development of K3”

#527

Post by GuB-42 »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

> Distillation is not illegal by every definition of the word.

I am waiting for a precedent on this one. In general, training on copyrighted material is legal, there is a lot of precedent there. But every now and then there is a case where the owner of the training material wins.

I don't remember the details but I believe one of these instances was when one company trained its AI on the knowledge base of another company and turned it into a competing product. Fair use was denied because of that direct competition. Distilling a LLM to make a competing LLM looks kind of like this, or maybe not, I don't know.

It would make sense for distillation to be legal in every way, LLMs are built on a broad interpretation of fair use, but sometimes, law is weird.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#528

Post by watwut »

linkregister wrote: post #496
noja wrote: post #171

Earlier quoted context omitted.

Isn’t that the same argument they are making for replacing human labour? Circumventing costs.

Do you get mad at your computer for replacing clerical workers? What does this nonsense comment have to do with the issue at hand?

Those were told "find another job" and in fact they were able to find different jobs.

AI companies are gleefully bragging and "making humans obsolete", "permanent underclass" and 40% unemployment rates they plan to create.

They pushed to replace people years BEFORE their technology even can produce that work.

So, you know, it is not the same. But also in fact, clerks did disliked when occasionally arrogant claimed to replace them while pushing unfinished software that dont quite work yet.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#529

Post by torginus »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

> The only argument they have here is that they use GB300 GPU's

I don't follow events closely, but the US has constantly flipflopped on what sort of GPUs the Chinese are allowed to have, not in small part because much of the AI boom's valuation is based on demand for US-made hardware, for which the Chinese have inexhaustible and well-financed demand.

So even this feels a bit hypocritical to me, but my understanding is that Chinese native AI hardware is getting good enough that labs dont feel a huge disadvantage by being forced to buy at home, even if they'd have preferred to buy US chips.

Which is a situation that was manufactured by the constant thread of having their access to advanced GPUs revoked.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#530

Post by tristanj »

TonyZYT2000 wrote: post #207
throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

I think the accusation implies Kimi has gained time travel capability (distilled from fable probably) to have enough time distilling fable. Given they can travel time now, I think it is fair to call them a threat to national security.

No, it's a flawed conclusion.

Claude Fable was publicly available for 72 hours early June. Moonshot more than enough time to prepare infrastructure, gather their preferred distillation data from Fable, and complete post-training well K3's mid-July launch.

Post reply on HN