honestly if they did what he said they did, it seems like it would be cheaper just to train your own model from the get go
“We have information that Moonshot distilled Fable for the development of K3”
61–70 of 742 posts
Re: “We have information that Moonshot distilled Fable for the development of K3”
#62Does this matter? Distillation is not illegal by every definition of the word.
There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them.
Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre-cursor acquisition.
And lastly, kimi architecture is vastly different than that of fable as it uses mechanisms developed by... kimi themselves. US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable.
Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.
edit: (moved this to bottom) The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#63I think they deserve, by Justice, to have their models pillaged and raped, just like they did to the internet. They didn't ask for permission when they took the entire of the internet, after all, and given their behaviour is nefarious, it's of Justice that they receive nefarious treatment by others, including chinese AI labs. The Chinese are not gonna deterred, but the posturing by the Americans is so blatantly hypoc…
I still believe taxing the bots, and implementing actual UBI, would address both problems.
Will the UBI apply to everyone worldwide? As that’s where the original dataset came from.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#64So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime
It's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an important point, and it's likely not the case.
It's also about the larger companies explaining why they can't be as efficient, of course they can't, they're not just ripping the outputs of another model that someone else invested billions to train.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#65You wouldn't distill a car.
distillation of wheat, barley and malt is delicious, though!
Re: “We have information that Moonshot distilled Fable for the development of K3”
#66If 'distillation' means training on outputs then what is the legal concept of ownership of outputs? And, more broadly, is this something that could be skirted by doing it in different countries that have different legal structures? Basically, are they saying they own those outputs, not the companies that paid for the tokens, and only they can train on them? I suspect a lot of companies are saving their token histories and using them to fine tune internal models.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#67Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies
It looks like these frontier-model companies don't really monitor their systems. Like OpenAI not realizing that it is their own AI which is attacking HuggingFace.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#68[deleted]
Re: “We have information that Moonshot distilled Fable for the development of K3”
#69Two recent ones that really really hit me,
> we're entering the most geopolitically volatile moment since the trinity test lit up the alamogordo desert and the only US policy prescription is a big button labeled sinophobia
https://bsky.app/profile/thebadcode.com/post/3mr3skoyass2k , and,
> every vendor cranking the big dial labeled "sinophobia" and looking back at the us government for approval
The government itself doing the propaganda here, skipping the vendors. Sinophobia intensifies. War drums of "be afraid be afraid be afraid" beat louder.
It's so bad, it's so stupid. Kimi lands one showing pretty clearly this was absolutely the determining concern happening at vast scale, that they can just a lot of this themselves, and this noise pollution from the most hopelessly lost aggro administration ever still gets blared out the trumpets of war & discord. What a joke. Give me a break, give it a rest.
War here is less winnable than the Iran war they started. They're going to make America itself so much worse, these people so hungry to put down free and good models. This pathetic attempt is not going to work, you are just going to once again hold the US citizens hostage & make their lives worse, for sick political games.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#70And?
Nobody cares. This is neither a controversy nor news, and that would be the case even if Anthropic hadn’t just settled a 1.5 billion dollar lawsuit where they trained Claude on thousands of books without permission lol.
To be clear I’m not taking a jab at OP - I’m saying the labs crying about distillation have neither a legal nor a moral leg to stand on. There’s nothing wrong with distillation.