Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

541–550 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#541

Post by 6gvONxR4sf7o »

JKCalhoun wrote: post #124
madduci wrote: post #55

So what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime

I'm by no means taking the side of the AI companies, but it's possible that Anthropic "added value" to the data they harvested. Stealing that does seem kind of uncool. Regardless, it was always inevitable—will continue to happen.

I wonder how the "added value" argument can apply to Anthropic and not to the Kimi team. If Claude's value is that you don't have to pay a team of slow expensive subject-matter experts, and Kimi's value is that you don't have to pay Claude, it just seems like the same thing.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#542

Post by culi »

throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

Yeah if anything Kimi's ability to distill that quickly is a major technological breakthrough

Re: “We have information that Moonshot distilled Fable for the development of K3”

#543

Post by slibhb »

bilbo0s wrote: post #472
slibhb wrote: post #328

Earlier quoted context omitted.

Of course it matters. Regardless of whether distillation is legal, there is a difference between training a model with and without distillation. For one thing, the distilled model wouldn't exist without the model it distilled. Also, companies that use distillation may be competitive but seem unlikely to surpass the companies that are training these models from scratch.

> but seem unlikely to surpass the companies that are training these models from scratch Then why is it a problem? Another serious question. Trying to get my head around what the root of the objection is here. There must be some fear, but if that fear is not a fear of being surpassed in the market, then what is the fear?

I didn't say it was a problem, I said it "mattered"

If I wanted to argue that it's a problem, I'd just say that companies investing billions in training frontier models should reap the rewards. And distillation is essentially theft.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#545

Post by dylan604 »

breppp wrote: post #533
dylan604 wrote: post #510

Earlier quoted context omitted.

so what? this has nothing to do with them taking from those that took before them. china censoring information they do not like is nothing new and seems irrelevant to this conversation

Fair enough you don't care, some people might want to know about the Uyghur happy camps, mass organ harvesting and such. In a world where information sources are only going to dwindle, it is not in anyone's interest to empower actors that will use these to manipulate perceptions

The people that care about that won't be using CCP products/services now will they?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#546

Post by DubiousPusher »

linkregister wrote: post #301

Commenters are overlooking the significance of this information and posting emotional reactions based on perceptions of fairness or feelings of schadenfreude. The economic viability of Anthropic and OpenAI rely on their being able to charge more for model access than their R&D and inference costs. If the market price for SOTA model access drops below that level, then these businesses will have to decide whether to co…

I think this is a really sober comment. There are lots of knock-on effects of this claim, even if it's not true which are consequential. The fact that a spokesperson for the US government is going out of their way to comment is concerning.

Strong bee-hive pinata vibes here.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#547

Post by tristanj »

skeledrew wrote: post #77

Super interesting. So Fable was really made available... a couple weeks ago? And K3 a few days ago? That's a really impressive feat to distill enough data AND train AND review to get a release that works really well in that time period. Mad props to the Moonshot team :flame:.

Fable was launched on June 9 (for 72 hours), then K3 launched on July 16.

Timeline wise, Moonshot had over a month to post-train K3 on Fable distilled data, which is more than enough time.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#548

Post by benjiro29 »

linkregister wrote: post #301

Commenters are overlooking the significance of this information and posting emotional reactions based on perceptions of fairness or feelings of schadenfreude. The economic viability of Anthropic and OpenAI rely on their being able to charge more for model access than their R&D and inference costs. If the market price for SOTA model access drops below that level, then these businesses will have to decide whether to co…

People keep forgetting that over the last 6+ months a lot of increased action has been taken by OpenAI and Anthropic to detect and combat distillation. Several are public known.

Combined with how short of a time Fable was around before K3 got released. I do not see how the data Moonshot is supposed to extract in such a short notice, that will enhance the model to such a point.

It sounds to me a lot of cope from the US, so they can give this as a reason to ban Kimi models from the market.

OpenAI/Anthropic their advantages used to be:

* Early growth advantage

* Access to a lot of client data to train upon

* Access to a lot of hardware to train upon

Several of those advantages have been eroded over time. That barrier has been shrinking. The US is not the only spot with a bunch of smart people (ironical seeing how many Chinese work in US R&D).

Thing is, even IF they distilled from Fable and got the model so trained up, it means that K3 is a base for future model development. The cat is already out of the bag with how good the model is. When the model gets released on the 27'th, any Chinese company will be able to train their models against K3 openly.

We are not in the past anymore, where DeepSeek was a unexpected hit, but where the Frontier models their advantages (compute, data, growth) prevented more Chinese models from growing.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#549

Post by Bratmon »

usef- wrote: post #534
trhway wrote: post #506

Earlier quoted context omitted.

The judge found their use is fair use. They are paying not for their use of the content, they are paying for using illegal copies of the content. The same principle can be applied to distillation - it is a fair use. You just shouldn't use illegal ways to access the models being distilled. To the commenter below: if it is illegal - has the police/FBI report been made? Otherwise it is just a civil court matter.

Fair, but isn't "illegal" access what they're talking about in OP? It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. It doesn't seem to be done by the international distillers. It's a cost that American open models will seem to have to pay but not international.

> It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making.

This is a very surprising claim to me (and I imagine many small website owners who keep getting scraped by Anthropic and OpenAI).

Do you have a source?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#550

Post by perching_aix »

Bratmon wrote: post #513
perching_aix wrote: post #431

Earlier quoted context omitted.

Yes, it's just a ToS violation at present. Those are legally binding though, despite the common adage. What that really translates to here though, anyone's guess. Anthropic's own copyright infringement could apparently be forgiven for 1.5B USD after all, so maybe there's a price that breaking the distillation clause for is acceptable too. Or some other arrangement.

But surely at least one of the websites Anthropic scraped to make Claude had a ToS forbidding automatic access? Why is Anthropic's ToS any more binding than that of a rabidly-anti-ai literature blog with 50 readers?

Do feel free to read the court documents to find out and let us know.

> Why is Anthropic's ToS any more binding than that of a rabidly-anti-ai literature blog with 50 readers?

Although I will say, this whole comparison stuff really doesn't seem to be your thing; might impede your analysis quite a lot: https://news.ycombinator.com/item?id=49013148

Maybe ask Claude?

Post reply on HN