How the tables have turned. It's okay for Anthropic to train their models on copyrighted data, but it's wrong to steal the stolen data from Anthropic models.
“We have information that Moonshot distilled Fable for the development of K3”
71–80 of 742 posts
Re: “We have information that Moonshot distilled Fable for the development of K3”
#72Is distillation something we have to live with or are there ways to prevent it?
You can prevent it by outputting a reasoning "summary" instead of the actual reasoning trace.
Which Anthropic already do.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#73I wonder if this points at a “shared” future (or at least things will eventually converge there whether companies like it or not). Ultimately, if you’re going to release these models that are fundamentally built on shared data - it’s pretty wishful to assume you’ll be able to harbor that model and the data, forever, and profit from it.
It also leads me to think about things like the original release of Fable 5, people were complaining that it was safeguarded too much - if you lock the models down too much they cease to be useful. So it’s going to be increasingly difficult to protect a model from competition while ALSO keeping it useful.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#74If you ask fable, it will identify as deepseek
Re: “We have information that Moonshot distilled Fable for the development of K3”
#75Hmm. I wonder when this was detected. And was the CoT trace cut from Fable from the start on June 9th or just after the export ban and relaunch? Is this what the export ban was actually about? I honestly don’t know, just wondering aloud.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#76Hah tales as old as time. what’s next? Distillation of Disney theme park?
Re: “We have information that Moonshot distilled Fable for the development of K3”
#77Super interesting. So Fable was really made available... a couple weeks ago? And K3 a few days ago? That's a really impressive feat to distill enough data AND train AND review to get a release that works really well in that time period. Mad props to the Moonshot team :flame:.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#78Is distillation something we have to live with or are there ways to prevent it?
Realistically you can't prevent distillation. OpenAI / Anthropic are slowly moving towards hiding the steps in-between input and output (hidden thinking), but that only helps so much. Imagine you put a file into Claude and say "do X to this" and it returns it to you without showing any of its internal reasoning. That's harder to distill, but the simple mapping of input to output still creates very valuable training d…
You can also get it to think in the output tokens pretty easily, eg "Here's a math problem, I want your reasoning first, then the answer" which is what I assume they're doing.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#79For code can't they distill from public GitHub commits? If they could figure out who used Mythos/Fable assitance in the commits.
Re: “We have information that Moonshot distilled Fable for the development of K3”
#80Earlier quoted context omitted.
I demand my right to pay 3x more for AI access. cf https://www.reddit.com/r/codex/comments/1uyj6pq/kimi_k3_is_1...
But nobody really pays the 3x (ie. api) rates, except for enterprises. Everyone else are using the consumption plans, which are heavily discounted[1], possibly cheaper than even the chinese models, which don't do consumption plan discounts. Even in your linked reddit thread, the OP agreed with this sentiment. [1] https://x.com/SemiAnalysis_/status/2064815044085318040
For now. We've seen this pattern play out literally a hundred times in tech and you're incredibly naive if you think this will last forever. And what is your point? It should be okay for US consumers if the US government illegalizes accessing open-weight models within their borders because they're currently getting a subsidized token rate?