Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

501–510 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#501

Post by breppp »

giaour wrote: post #497
breppp wrote: post #467

Earlier quoted context omitted.

Because model output is probably far closer to software or a licensed work which possibly has greater protections than it is to copyright. There is far less possibility of fair use, it might be protected by patents, license or reverse engineering laws. In any case the laws are being written now, but I doubt these will have worse protection than software does, which has far better protections than copyright

> I doubt these will have worse protection than software does, which has far better protections than copyright Software is protected by copyright. Some software may also be protected by patents, but last time I checked, AI generated output of any kind was not patentable.

Software is protected by the DMCA, patents, licenses, EULAs, all of those aren't there for books. I doubt new laws won't be written for model outputs.

Also, if model output distillation is shown as some form of reverse engineering I assume the DMCA can apply

Re: “We have information that Moonshot distilled Fable for the development of K3”

#502

Post by trhway »

joshuamorton wrote: post #366
Bratmon wrote: post #355

Earlier quoted context omitted.

I like this comment because its argument only makes sense if you assume that the entire world's output of books and art did not require a huge amount of resources and expertise to make, nor did it add any value. It's the most CS-major take ever!

I don't think that's what it's saying at all. It's saying that there's a level of creativity in model creation that isn't present in distillation.

>a level of creativity in model creation that isn't present in distillation.

the same argument - a level of creativity in the world knowledge creation that ins't present in the model training on that knowledge.

Or in other words - model creation and training is just a distilling of the world knowledge.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#503

Post by sailingparrot »

qeternity wrote: post #349
MiguelVieira wrote: post #110

Here's a site that asks the same questions to 22 models and compares how similar their responses are. https://typebulb.com/u/lab/you-re-relatively-right/full According to these results GLM 5.2 is very similar to Google Gemini and Kimi K3 is very similar to Fable 5. The American frontier labs are not similar to each other.

I think it’s fairly obvious the Chinese labs are doing mass distillation. I also think the Fable accusation is wrong and it was most likely Opus 4.8 which itself is likely a distillation of Fable.

You have it backwards IMHO, obviously can't prove it, but I would bet that Opus was used to bootstrap Fable. It becomes very confusing since we have started calling everything distillation, but most likely what both Anthropic did for Fable and Moonshot did for K3 was using Opus traces in the reasoning SFT stage during mid training.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#504

Post by phs318u »

Do I, as a paying user, own the output the LLM has produced or am I merely licensing the output?

Most paying users assume ownership, in which case I’ll do with that output what I want.

If the LLM outputs are not owned by the user, but are actually licensed, please clarify the terms of commercial use.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#505

Post by joshuamorton »

trhway wrote: post #502
joshuamorton wrote: post #366

Earlier quoted context omitted.

I don't think that's what it's saying at all. It's saying that there's a level of creativity in model creation that isn't present in distillation.

>a level of creativity in model creation that isn't present in distillation. the same argument - a level of creativity in the world knowledge creation that ins't present in the model training on that knowledge. Or in other words - model creation and training is just a distilling of the world knowledge.

I don't disagree. I'm not sure why that's a relevant reply though.

If you think that the addition of a less creative process (model creation) to a more creative corpus ("art") is problematic, then it follows that you should think the addition of a less creative process (distillation) to a more creative corpus (a model) is also problematic.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#506

Post by trhway »

usef- wrote: post #480
foo12bar wrote: post #398

Earlier quoted context omitted.

The models were built using copyrighted works, so why can't models be built using other models?

They do seem to be paying for it (as per the 1.5Bil lawsuit yesterday and them now purchasing books and licensing from media companies). Whether we think they're paying enough is another question, but "I'm paying for content so can protect it" doesn't seem inconsistent. We may decide that giving models away for free means they don't have to license content (judging by HN comments), but currently that doesn't seem to…

The judge found their use is fair use. They are paying not for their use of the content, they are paying for using illegal copies of the content.

The same principle can be applied to distillation - it is a fair use. You just shouldn't use illegal ways to access the models being distilled.

To the commenter below: if it is illegal - has the police/FBI report been made? Otherwise it is just a civil court matter.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#507

Post by Barrin92 »

remus wrote: post #265
jaggederest wrote: post #142

Earlier quoted context omitted.

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…

>Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way.

producing the entire body of human knowledge that Silicon Valley companies absorbed like the Borg did not just take more resources but also a fair amount of blood and sweat, certainly more than the LLM so on that front that comparison also seems entirely justified.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#508

Post by program_whiz »

To borrow the argument from the apologists:

But everyone learns by example! How is this any different from a person just reading the outputs of Fable, learning, then producing output. Surely reading outputs, gaining knowledge, then producing work isn't illegal, or all art/writing would be illegal.

Funny how that argument seems so vacuous in this situation, yet others find it compelling when justifying the mass theft of art and writing for model creation. In this case the model is "just learning priors" before it "creates its output which is novel", nothing problematic.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#509

Post by giaour »

breppp wrote: post #501
giaour wrote: post #497

Earlier quoted context omitted.

> I doubt these will have worse protection than software does, which has far better protections than copyright Software is protected by copyright. Some software may also be protected by patents, but last time I checked, AI generated output of any kind was not patentable.

Software is protected by the DMCA, patents, licenses, EULAs, all of those aren't there for books. I doubt new laws won't be written for model outputs. Also, if model output distillation is shown as some form of reverse engineering I assume the DMCA can apply

You may recall that the DMCA was originally written to protect music and movies. It does in fact apply to creative works. If you have ever purchased an MP3, eBook, or streaming movie, you will also be aware that you purchased a license to the underlying IP. This is also true of physical media, but the license agreement you have to accept when obtaining a digital work makes this explicit.

I agree that you can't patent a book, but I would point out that you can patent an idea, which may only appear in a book or journal article.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#510

Post by dylan604 »

breppp wrote: post #492
bilbo0s wrote: post #469

Earlier quoted context omitted.

Why? Serious question. I'm from the US, and I think it's hilarious.

One example is that a totalitarian government known for erasing historical record of its crimes will control an arbitrator of truth.

so what? this has nothing to do with them taking from those that took before them. china censoring information they do not like is nothing new and seems irrelevant to this conversation
Post reply on HN