Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

281–290 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#281

Post by Daishiman »

kevinqi wrote: post #258
mring33621 wrote: post #181

Earlier quoted context omitted.

People that think the Chinese are only able to copy western tech are in for a wakeup call. Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.

my assertion isn't that china isn't able to surpass western AI. I think it may well happen. I've been to china many times and am well aware of how ahead they are in many technological/societal areas. at the same time, I don't buy the idea that distillation is unimportant in assessing what Chinese labs are capable of. If it wasn't, why did Kimi's release timing coincide so well with Fable's launch? and if Anthropic ha…

> and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider.

That works both ways, competition and performance spur new developments. You don't think the American labs are looking at Chinese research on how to reduce compute per token?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#282

Post by nylonstrung »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

I wouldn't be surprised at all if US labs are also distilling Chinese models, except we'd never know since they can simply self-host them

Re: “We have information that Moonshot distilled Fable for the development of K3”

#283

Post by skippyfish »

remus wrote: post #265
jaggederest wrote: post #142

Earlier quoted context omitted.

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…

> raining a SOTA model takes a huge amount of resources and expertise

Writing books, building Wikipedia, and answering questions on online forums takes a lot of resources and expertise that scraping didn't. So at the very least, we're already one rung down the "maybe you should've asked" ladder.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#285

Post by Gud »

OK, DIRECTOR Michael Kratsios, but why should we give a shit?

American AI corporations are pushing up the prices for computing, making it unaffordable for the common man. Additionally, they have built their entire business on stealing(yes, stealing) work from us.

So fuck em

Re: “We have information that Moonshot distilled Fable for the development of K3”

#286

Post by chasd00 »

gensym wrote: post #239
chasd00 wrote: post #226

if there are no consequences then who cares? You're not going to take Chinese companies to court and stealing IP is nothing new either. It's going to take some sort of policy change at the federal government level to do anything but they haven't done much up to this point. Maybe AI is important enough to actually get some kind of policy change, sucks for everyone else who have had their IP stolen with no consequences…

I fear they are laying the groundwork to ban US citizens from using Chinese models, so that they can make sure that Brockman gets his money's worth.

why would you fear that? At least it's something to fight unfair business practices. As for American AI companies violating copyright there's a venue for that, the courts. File a case if you feel your copyright has been violated and get your day in court.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#287

Post by trollbridge »

softwaredoug wrote: post #166
madduci wrote: post #55

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime

If they did this in the US they would almost certainly be sued. Meta, for examples, doesn’t want employees to use Claude Code due to distillation risk.

It turns out U.S. law doesn’t have jurisdiction across the entire world, nor does Anthropic and OAI’s rather blatant attempt to buy government influence.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#288

Post by nylonstrung »

kevinqi wrote: post #178
himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.

So many of the breakthroughs and architecture that make LLMs powerful in general today came from China, especially ones related to sparsity and MoE that have made inference and training substantially cheaper.

Let's not forget how much people talked about "prompt engineering" before Deepseek mainstreamed the idea of thinking mode which is now universal

Re: “We have information that Moonshot distilled Fable for the development of K3”

#290

Post by trollbridge »

azinman2 wrote: post #270
Lalabadie wrote: post #115

Earlier quoted context omitted.

"You are trying to kidnap what I have rightfully stolen!"

Except it’s not just a dump of the internet, which Moonshot also did themselves (and probably used even more pirated content as laws in China are different without any recourse for the entire world). I don’t know why this is so unclear to folks.

Chinese IP law is actually quite solid. You do have to register your trademarks and copyrights properly in China, and then lawsuits have to be filed appropriately according to Chinese law. Is that a problem?
Post reply on HN