Presumably the frontier labs themselves can do their own distillation far better than the chinese labs can. Why can't they just beat them at their own game and release / host low cost intelligence and own the whole game. There will always be a market for the more expensive frontier intelligence.
“We have information that Moonshot distilled Fable for the development of K3”
161–170 of 742 posts
Re: “We have information that Moonshot distilled Fable for the development of K3”
#162rules for thee but not for me
Plenty of HN readers feel this way and it's a good point, but it has also become an entirely cliché response which pops up like mushrooms anytime "distillation" appears. That means it's against the site guidelines, which ask: " Eschew flamebait. Avoid generic tangents. Omit internet tropes. " - https://news.ycombinator.com/newsguidelines.html I don't mean to pick on you personally! It's just that reflexive responses…
You just told on yourself big time about being a lapdog for the US Feds. Easily one of the worst moderation decisions you’ve ever made and that’s impressive given your track record.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#163Earlier quoted context omitted.
Uh, what? > Distillation is not illegal by every definition of the word Note that Anthropic (and USG) alleges [0] not only that Kimi was distilled, but that they actively circumvented measures intended to stop distillation. There are multiple ways that's illegal, including: - Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done. - Economic espionage: 18 U.S.C. §1831…
All of the models stole the entirety of written knowledge on the internet to train. They are being sued for the few cases where we have some proof of what they did because of some whistleblowers, all the rest will just go unpunished. They breached Github TOS, robot.txt's, copyright, patents every form of IP protection under the sun from a billion sources. It's just ridiculous for the thieves to cry about someone else…
Do you care about the law or not? I think theft is bad everywhere, not just when Anthropic does it.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#164Is distillation something we have to live with or are there ways to prevent it?
If you build a device that can help build devices, you shouldn't be too surprised when people use it to build devices.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#165Two recent ones that really really hit me, > we're entering the most geopolitically volatile moment since the trinity test lit up the alamogordo desert and the only US policy prescription is a big button labeled sinophobia https://bsky.app/profile/thebadcode.com/post/3mr3skoyass2k , and, > every vendor cranking the big dial labeled "sinophobia" and looking back at the us government for approval The government itself…
Not surprising that Bluesky is perpetually in peak woke mode, but the racism claims are absurd. If Russia were doing the same thing, would we have no problem with that because they're white?
Re: “We have information that Moonshot distilled Fable for the development of K3”
#166So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime
If they did this in the US they would almost certainly be sued.
Meta, for examples, doesn’t want employees to use Claude Code due to distillation risk.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#167Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…
It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other leveraged companies are at risk.
The reason why the United States government is weighing in is because it's in the national interest of the US to have supremacy in "AI".
Legality or lack thereof is one of many data points about whether a thing is noteworthy.
Moonshot performing distillation is rational from their point of view. Reducing costs is in the interest of businesses. It's also rational for frontier labs and the US government to add obstacles to this process.
As consumers this is probably a positive development.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#168Earlier quoted context omitted.
This is BS to pressure politicians. Even an openai's guy (head of something made up) called bs on the idea you can train something like k3 by distillation. Anybody I know who works in LLM research says that distillation is either useless or merely useful in post training to show "correct" behavior. And even then you don't get a competing model, if RL on good prompts was that useful, all labs would've long skyrocketed…
Dean Ball, "head of strategic futures" at openai. https://xcancel.com/deanwball/status/2078133895766114412#m
> One probable outcome of an open-weight-model-dominant world is full AI communism ... This future strikes me as a dystopian hellscape.
Wow, just wow. He is not even subtle about it.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#169I can understand that the AI labs might care about other labs distilling their models as it can eat into their competitive advantage, but do consumers care at all? Aren't consumers benefiting from this practice by getting better cheaper models as a result?
They're probably going for the national security/domestic manufacturing angle. > Aren't consumers benefiting from this practice by getting better cheaper models as a result? Aren't consumers benefiting from cheap chinese batteries, EVs, and drones?
I am not sure if this is implicitly part of the point you meant to make, but I just wanted to point out for those who aren't aware that Chinese EVs and drones are both banned in the US on precisely the (vague) grounds you mention. The drone ban is more recent and nominally only affects new models that haven't yet received FCC certification, but the outcome if nothing changes will be that American consumers lose access to DJI-style videography drones. DIY hobbyists may also find it more difficult or impossible to source parts for their projects.
Routers have now gotten the same treatment. So yes, consumers have been historicaly benefiting from all these things, and those benefits are about to evaporate as we lose access to cheap and high quality Chinese products before any domestic equivalents exist. And IIUC banning the use of Chinese LLMs for consumers and/or businesses in the US is now being discussed at the highest levels of government, with the "encouragement" of US AI firms.
I don't think any of these people care that America consumers are increasingly going to feel like they're living in a sanctioned country. It's all about the defense and b2b segments.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#170Earlier quoted context omitted.
I think we have some historical precedents for this that went less well than you might hope.
I am not aware of any revolutions where an international solidarity of the working classes overthrew their capitalist exploiters. I am aware of a couple of national solidarity movements which overthrew a monarchy (France; Russia) and a dictatorship (Cuba), but no international ones.
I think it’s safe to extrapolate.