51–60 of 73 posts
This data sort of disqualifies itself: unless Moonshot has a time machine, K3 should be more similar to Opus 4.5-4.8 than Fable 5. Keep in mind, Anthropic started limiting access and introduced anti-distillation measures around 4.5-4.6 (?). So the majority of distillation should have happened on earlier models. Maybe a better explanation is that they have access to the same training datasets? Which if private can aga…
Earlier quoted context omitted.
What of the costs for creating the data that was used to train the model being distilled?
Do you people seriously believe that Moonshot and the Chinese personal-cult-state dont possess and train on the same torrents?
Earlier quoted context omitted.
Does distilling actually violate any IP protection laws? Sure, it’s against their terms of service.
I get what you're saying and two wrongs do not make a right but the irony, and why people are even talking about this, is that the thing being distilled clearly, and knowingly, violated copyright & terms across the entire internet.
All frontier models have been trained without any regards for IP protection laws. I don't see how anyone can argue in good faith that distillation is not fair game and does not ultimately "benefit humanity™"
For me, I don't really care about the theft aspect but when people are claiming that these open models are better value or going to overtake anthropic/openai models, the implication that the open models are training of distilled data means all the "progress" they are making is just mimiced from the closed models. It's a bit interesting how the open models are able to keep pace with the closed models except whole main…
All frontier models have been trained without any regards for IP protection laws. I don't see how anyone can argue in good faith that distillation is not fair game and does not ultimately "benefit humanity™"
Does distilling actually violate any IP protection laws? Sure, it’s against their terms of service.
All frontier models have been trained without any regards for IP protection laws. I don't see how anyone can argue in good faith that distillation is not fair game and does not ultimately "benefit humanity™"
For me, I don't really care about the theft aspect but when people are claiming that these open models are better value or going to overtake anthropic/openai models, the implication that the open models are training of distilled data means all the "progress" they are making is just mimiced from the closed models. It's a bit interesting how the open models are able to keep pace with the closed models except whole main…
The interesting question is: will Anthropic release a Fable like model with an architecture similar to Kimi, and get the inference cost gains? They should surely beat Kimi because they can internally distill as much as they want.
All frontier models have been trained without any regards for IP protection laws. I don't see how anyone can argue in good faith that distillation is not fair game and does not ultimately "benefit humanity™"
Training a model is very expensive and creates something no individual rights-holder could. Distilling a model copies this value add and captures it without bearing the cost that created it.
Earlier quoted context omitted.
What of the costs for creating the data that was used to train the model being distilled?
At least 1.5B by stealing books, per a recent ruling. I don’t feel sorry for the model companies
Earlier quoted context omitted.
What of the costs for creating the data that was used to train the model being distilled?
Do you people seriously believe that Moonshot and the Chinese personal-cult-state dont possess and train on the same torrents?