Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

431–440 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#431

Post by perching_aix »

skybrian wrote: post #423
joshuamorton wrote: post #366

Earlier quoted context omitted.

I don't think that's what it's saying at all. It's saying that there's a level of creativity in model creation that isn't present in distillation.

Maybe, but it's not like their AI is likely to repeat it back verbatim so it's unlikely to be a copyright violation. It seems like at most, they would be breaking Anthropic's terms of service? Or maybe they're going through an intermediary "transfer station" that's breaking terms of service: https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...

Yes, it's just a ToS violation at present. Those are legally binding though, despite the common adage. What that really translates to here though, anyone's guess.

Anthropic's own copyright infringement could apparently be forgiven for 1.5B USD after all, so maybe there's a price that breaking the distillation clause for is acceptable too. Or some other arrangement.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#433

Post by bluegatty »

tikhonj wrote: post #380
bluegatty wrote: post #375

Earlier quoted context omitted.

No - distillation is not data inputs. Raw materials vs. Value add. They are different things, like ore and metal. Distillation is a new thing we need to understand, it's probably closer to IP than not.

The "data inputs" were also, very much, somebody's "value added" IP. We're talking about things like text people wrote , not some kind of raw data floating out in the ether.

Did I say there was no value add in the inputs?

Ore has value, a different kind of value than the output of the refinery.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#435

Post by bluegatty »

blks wrote: post #407
bluegatty wrote: post #379

Earlier quoted context omitted.

This is a misrepresentation though. The LLM output, is not the same as the input - there is value add. Of course works used as raw inputs to LLMs required work and are reasonably subject to IP concerns - but they are different. It's possible that the LLM makers 'owe' the content creators that created the content they used to make their products - it's an interesting but separate question. We could very well end up wh…

Lossly storing IP in LLM itself, and using IP for training (so it’s lossly stored in LLM), without licensing these works or otherwise following license agreements (eg GPL) is infringement. Using then this product for commercial activity is a smoking gun.

"Lossly storing IP in LLM itself, a" - that part I'm inclined to agree with.

But it's debatable if that's the case.

Google stores copyrighted content and produces in in their product.

Also - it's fair game to use snippets of things here and there, if the derived work is novel, which I think it is for LLMs, mostly.

I do agree though, that we ought to draw the line somehow.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#436

Post by XorNot »

himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

It certainly matters as familiar sounding words to their stock holders to please not drop the valuation.

Because what they want them to think is "the AI factory has unique proprietary technology that cannot be replicated"

What they don't want them to think is "it's relatively easy once you know the basics to bootstrap to near SOTA and so the commercial case for selling inference has an extremely short profitability horizon with little if any brand loyalty or lock in".

Re: “We have information that Moonshot distilled Fable for the development of K3”

#437

Post by overgard »

kevinqi wrote: post #178
himata4113 wrote: post #62

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…

I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.

I think it depends on where you think we are on the S curve of intelligence growth. (Yes, I think it's an S curve, not an unbounded exponential). If you think we're near the peak than playing catch up (especially if you can play catch up quickly) is very rational.

I know this isn't exactly a scientific test, but I had a local Qwen 3.6 27B model implement a fairly sizable feature today. There were a couple of bugs, mostly around me not giving sufficient specifications, but they were ironed out quickly when I pointed it out. I was able to ask the model to create instructions so next time it doesn't fall into the same pitfalls, and it did a great job. 27B local model! (And it was super fast too).

I ran Fable 5 as a code review and it didn't really have any significant corrections.

I guess my point here is that, for most work the frontier models are probably overkill anyway, and improving on overkill in a way that raises prices significantly is probably not a winning strategy.

The only place I can think of where the super high powered models are "required" is if you want to do a ridiculous token burn like GasTown where you just have it run un-monitored on very long tasks. To me though, that's an experiment, not a real workflow. And the way these labs are like "oh we made this (broken) thing in a week using just agents!" always also follows with "and it cost $100,000+ in tokens!". Like, ok, I get it if you're doing research but that's the salary of an entire person.. that can actually learn and improve.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#438

Post by owebmaster »

azinman2 wrote: post #267
dylan604 wrote: post #254

Earlier quoted context omitted.

This is why I don't give a shit that this is happening. It's actually kind of funny to me.

Unless you’re from mainland China, you should.

Why so? If I'm from Europe or South America, should I hope Anthropic/Openai win?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#439

Post by bluegatty »

jaggederest wrote: post #390
bluegatty wrote: post #379

Earlier quoted context omitted.

This is a misrepresentation though. The LLM output, is not the same as the input - there is value add. Of course works used as raw inputs to LLMs required work and are reasonably subject to IP concerns - but they are different. It's possible that the LLM makers 'owe' the content creators that created the content they used to make their products - it's an interesting but separate question. We could very well end up wh…

> but they are different. How, and why? > We could very well end up where content IP is protected, LLM output is not and visa versa with reasonable legal founding, doubtful but plausible. That is the current state of legal rulings - LLM output is public domain, not copyrightable.

"> but they are different.

How, and why?"

How are they even remotely the same?

They're not even used the same way.

One is raw data input, the other is training content - designed to train LLMs.

One is a set of IP derived for other purposes entirely, and has esablished IP law - how you can use someone else's creative work or not ... for LLM outputs, less clear.

Post reply on HN